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SUMMARY  OP  STUDY  RESULTS 


1.1  Introduction 

-'The  objective  of  this  study  was  to  demonstrate  the  use 
and  applicability  to  Air  Force  software  acquisition  managers  of 
six  quantitative  software  reliability  models  to  a  major  command, 
control,  communications,  and  intelligence  (Cgl)  system.  The 
scope  of  the  effort  involved  the  collection  of  software  error 
data  from  an  ongoing  C^i  project>  (the  Hughes  Joint  Surveillance 
System,  JSS,  was  selected) , -fitting  the  six  models  to  the  data 
thus  collected,  analysis  of  the  predictions  provided  by  the  mod¬ 
els,  and  the  development  of  conclusions,  recommendations,  and 
guidelines  for  software  acquisition  managers  pertaining  to  the 
use  and  applicability  of  the  six  software  reliability  models.- 

This  research  was  partially  motivated  by  the  recommenda¬ 
tions  from  the  Validation  of  Software  Reliability  Models  study 
(RADC-TR-79-1^7)  (Schafer,  et.al.  (1979)).  In  that  study  it  was 
determined  that  the  cause  of  the  generally  poor  fits  obtained  for 
the  models  studied  therein  could  not  be  conclusively  attributed 
to  the  failure  of  the  internal  assumptions  of  the  models,  and 
that  the  integrity  of  the  data  used  was  a  significant  factor 
whose  effect  could  not  be  determined.  It  was  recommended  that  a 
controlled  data  collection  project  be  undertaken  to  assure  data 
integrity  and  therefore  provide  better  means  for  evaluating  the 
models . 

While  a  C^l  project  does  not  afford  the  degree  of  con¬ 
trollability  implicit  in  the  recommendation  of  the  Validation  of 
Software  Reliability  Models  study,  every  effort  was  made  in  this 
investigation  to  collect  the  highest  quality  data  possible  with¬ 
out  significantally  perturbing  the  JSS  project.  In  cases  where 
the  exact  data  input  requirements  for  a  model  were  not  met,  we 
were  able  to  use  the  models'  assumptions  to  define  a  version  of 
the  model  which  would  accept  the  data  available.  In  other  in¬ 
stances,  the  manner  in  which  the  data  was  collected  was  known  a- 
priori  to  be  in  violation  of  a  models'  assumptions.  In  these 
instances,  we  were  able  to  use  supplementary  data  (such  as  test 
phase  and  compilation  unit  name)  to  restrict  attention  to  a  class 
of  data  in  which  the  assumptions  would  be  roughly  satisfied. 
Overall,  every  effort  was  made  to  improve  the  fit  of  each  model, 
without  altering  data  (e.g.,  "estimating"  the  times  between  error 
detections  rather  than  observing  them) . 

In  performing  this  study,  we  have  tried  to  formulate 
guidelines,  conclusions,  and  recommendations  based  on  the  results 
of  fitting  the  models  to  the  JSS  data.  It  is  conceivable  (al¬ 
though  presumably  less  so  on  other  Hughes  (PI  projects)  that  dif¬ 
ferent  conclusions  could  be  obtained  from  data  from  a  different 
CPI  project.  A  wide  range  (from  no  success  to  complete  success) 
of  results  have  been  reported  in  the  past  by  other  researchers 


for  the  models  studied  herein  based  on  different  data  sources. 

We  are  not  aware,  however,  of  any  past  efforts  performed  on  C^l 
data  as  extensive,  well  controlled,  and  well  documented  as  the 
JSS  data.  Nevertheless,  because  the  software  error  detection  and 
removal  process  is  so  dependent  on  factors  extraneous  to  the 
software  (e.g.  testing  intensity,  manpower,  skill  levels, 
scheduling  constraints,  etc.)  we  caution  that  extrapolation  of 
our  conclusions  to  other  projects  in  other  companies  may  be 
spurious . 

It  is  hoped  that  the  results  of  this  study  will  provide 
assistance  to  RADC,  and  other  government  agencies  involved  in 
research  and  development,  in  directing  their  future  resources  in 
software  reliability  modeling.  It  is  also  hoped  that  the  data 
collected  in  this  study  can  be  used  by  other  researchers  as  a 
"proving  ground"  for  any  new  models  which  may  be  proposed  for 
future  industrial  use . 

1 • 2  Data  Collection 

The  primary  motivation  for  collecting  software  error 
data  is  to  improve  the  quality  of  the  delivered  system.  These 
data  are  valuable  not  only  to  assess  and  predict  the  quality  of 
the  software  from  which  they  were  gathered,  but  also  to  provide 
lessons  learned  for  the  next  similar  project.  Often  errors  are 
repeated,  even  with  the  same  programming  staff,  on  similar  proj¬ 
ects  merely  because  of  the  complexity  of  the  software  and  the 
inability  to  remember  the  details  of  past  experience  (Gannon, 
1983). 

It  is  imperative  that  both  project  and  data  collection 
personnel  be  trained  in  the  definition  of  error  classifications, 
as  well  as  the  data  collection  procedures.  This  training  should 
take  place  before  software  development  begins.  Automated  data 
collection  may  be  the  only  means  to  obtain  objective  data,  but 
some  projects  either  cannot  afford  the  extra  expense  or  are  pre¬ 
cluded  from  utilizing  automated  schemes  because  of  security 
restrictions.  It  is  essential  that  accuracy  and  consistency  of 
the  data  be  validated  early  in  the  project  as  well  as  throughout 
the  duration  of  the  project. 

The  following  is  a  summary  of  guidelines  for  the  collec¬ 
tion  of  error  data  from  software  development  projects,  for  the 
purpose  of  providing  input  data  to  software  reliability  models 
and  metrics: 


o 


Early  training  in  classification/collection 
procedures 


o  Standard  classification  and  consistent 

definitions 

o  Collection  should  start  when  software  is 

under  configuration  control 

o  Continual  monitoring  including  automatic 

validation  and  troubleshooting 

o  Use  of  automated  procedures  for  recording, 

qualification,  and  reduction 

1.3  Models  and  Usage  Guidelines 

The  six  models  under  investigation  in  this  study  were 
the  Geometric  Poisson  (Moranda,  1975) »  Nonhomogeneous  Poisson 
(Goel  &  Okumoto,  1980),  Imperfect  Debugging  (Goel,  1978), 
Generalized  Poisson  (Schafer,  et.al.  1979).  IBM  Poisson  (Brooks  & 
Motley  1980),  and  Binomial  (Schafer  et.al.  1979).  In  studying 
these  models,  many  striking  similarities  with  the  original 
Jelinski-Moranda  (Moranda,  1975)  model  and  its  principles  were 
discovered.  In  fact,  it  was  found  that  for  purposes  of  compari¬ 
son,  each  model  could  be  reparametrized  in  terms  of  parameters 
which  directly  relate  to  the  Jelinski-Moranda  parameters  (namely, 
N,  the  initial  number  of  errors,  and ,  the  error  detection  rate 
of  a  single  error) . 

Because  of  the  unavailability  of  the  exact  times  between 
error  detections,  it  was  necessary  to  modify  the  procedure  for 
fitting  the  Imperfect  Debugging  Model  (Goel,  1978).  It  is  shown 
in  Section  3-2  that  if  only  errors  which  are  actually  removed  are 
counted  in  the  process,  then  the  Imperfect  Debugging  model  re¬ 
duces  to  the  original  Jelinski-Moranda  model.  Based  on  this  ob¬ 
servation,  a  version  of  the  Jelinski-Moranda  model  designed  for 
frequency  data  proposed  by  Lipow  (197^)  and  further  developed  by 
Schafer  et.al.  (1979)  is  used  in  place  of  the  Imperfect  Debugging 
model . 


Another  model  which  required  modification  was  the 
Geometric  Poisson  Model  (Moranda,  1975)-  The  modification  re¬ 
quired  was  to  allow  unequal  time  interval  lengths  in  the  input 
data.  This  modification  was  straight-forward,  and  made  entirely 
within  the  assumptions  of  the  original  equal  time  interval  model. 

The  final  model  which  was  modified  was  the  IBM  Poisson. 
Although  this  modification  was  not  strictly  necessary,  it  was 
made  because  we  felt  it  would  improve  the  model’s  fit.  The 
original  version  was  also  fitted,  in  addition  to  the  modified 
version.  The  modified  IBM  model  is  discussed  in  Section  3-^- 


The  results  of  fitting  the  models  are  given  in  Section 
The  procedure  used  to  fit  the  models  was  to  estimate  the 
models’  unknown  parameters  using  the  techniques  advocated  by 
their  respective  authors  (usually  a  pseudo  maximum  likelihood  or 
pseudo  least  squares  principle).  A  chi-square  goodness-of-f it 
test  was  then  performed  whenever  valid  parameter  estimates  were 
obtained.  The  data  selected  for  fitting  the  models  was  chosen 
from  single  compilation  units  (CUs)  and  test  phases  to  eliminate 
the  effects  (which  are  counter  to  the  assumptions  made  by  the 
models)  of  software  build-up  and  variability,  and  varying  test 
intensity. 

In  general,  the  models  performed  poorly  with  respect  to 
fit.  Each  model  experienced  problems  with  lack  of  convergence  in 
its  parameter  estimation  algorithms.  Nevertheless,  the  results 
were  considerably  better  than  those  achieved  in  the  Validation  of 
Software  Reliability  Models  study  (Schafer,  et.al.  1979)  with  the 
best  fitting  model  overall  being  the  modified  version  of  the  IBM 
Poisson  model  with  53%  of  its  attempts  leading  to  a  good  fit 
(i.e.  a  good  fit  means  that  the  chi-square  goodness-of-f it  test 
was  not  failed  at  the  0.05  level  of  significance).  A  summary  of 
the  results  is  given  in  Table  1.3.1. 

A  measure,  derivable  from  the  outputs  of  any  of  the  six 
models  studied  herein,  which  would  be  of  use  in  monitoring  formal 
and  qualification  testing  of  software,  is  introduced  in  Section 
3.11.  This  measure  is  the  residual  number  of  errors  in  the  soft¬ 
ware.  In  comparing  this  measure  as  provided  by  the  software 
reliability  models  studied  herein  with  the  actual  performance 
history  of  the  JSS  project,  an  important  observation  can  be  made. 
First,  the  models  (when  they  fit)  provide  little  information  con¬ 
cerning  the  number  of  errors  in  future  test  phases.  That  is, 
estimates  of  residual  errors  based  on  current  data  are  inconsis¬ 
tent  with  the  number  of  errors  subsequently  detected  and  removed 
in  the  next  test  phase.  We  believe  that  this  is  best  explained 
not  only  by  possible  inadequacies  in  the  models,  but  mostly  by 
the  fact  that  each  test  phase  can  expose  only  its  own  class  of 
errors,  and  some  of  these  errors  may  be  uniquely  detectable  by 
that  test  phase  and  no  other. 


Table  1.3.1 


Summary  of  Model  Fitting  Results 


%  lack  of 


Model 

*  fit 

%  no  fit 

converaence 

Modified  IBM 
Poisson 

53 

22 

25 

Nonhomogeneous 

Poisson 

33 

25 

42 

Geometric 

Poisson 

33 

25 

42 

Generalized 

Poisson 

35 

10 

55 

Binomial 

27 

16 

57 

Jelinski- 

Moranda 

18 

8 

74 

IBM  Poisson 

0 

0 

100 

! 


i 


i 


i 


i 
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While  the  general  recommendation  concerning  the  models 
studied  herein  is  that  they  not  be  adopted  for  general  or  con¬ 
tractual  use  either  by  acquisition  managers  or  software  project 
managers,  some  guidelines  can  be  followed  which  will  aid  in  their 
use.  These  guidelines  are: 

a)  Collect  error  data  according  to  the  guide¬ 
lines  for  data  collection  in  Sections  1.2 
and  2.JJ. 

b)  Apply  the  models  at  the  compilation  unit 
level. 

c)  Apply  the  models  to  data  within  a  single  test 
phase . 

d)  Interpret  the  results  in  the  context  of  that 
test  phase  only,  and  use  the  results  to 
decide  if  more  testing  within  that  phase  is 
necessary. 

e)  Do  not  use  the  results  of  a  model  if,  in 
fitting  the  model,  it  falls  the  chi-square 
goodness-of-fit  test  at  an  appropriate  level 
of  significance  (we  recommend  0.05). 


2. 


DATA  COLLECTION 


5n  compliance  with  the  statement  of  work,  Hughes  uti- 
I  software  development  project  under  the  control  of  ESD 
for  the  demonstration  of  the  applicability  of  software  relia¬ 
bility  prediction  models.  Hughes  selected  the  Joint  Surveillance 
System  (JSS)  which  qualified  with  respect  to  the  following 
characteristics:  1)  development  IAW  AP  800-14,  Volume  II;  2) 
use  of  high  order  programming  language  IAW  DODI  5000.31;  3)  an 
estimated  size  of  at  least  20,000  DSLOC  excluding  comments;  and 
4)  a  development  schedule  compatible  with  the  demonstration 
study  schedule.  The  following  paragraphs  describe  JSS  project 
characteristics,  emphasize  lessons  learned  in  the  classification 
of  errors,  and  present  recommended  guidelines  for  error  data 
collection. 

2 . 1  Description  of  JSS  Project 

JSS  is  a  complex  air  defense  system  for  North  America. 
Seven  regional  control  centers  supported  by  86  sensor  sites  pro¬ 
vide  the  command,  control,  communications,  and  surveillance  func¬ 
tions  for  this  system.  The  system  provides  for  the  transfer  of 
sensor  data  from  the  sites  to  the  regional  control  centers,  the 
lateral-tell  of  track  and  status  information  between  centers, 
forward-tell  of  track  and  status  information  between  centers,  and 
the  forward-tell  of  all  information  from  the  centers  to  a  central 
operations  center.  The  system  is  capable  of  operating  in  stan¬ 
dard  and  degraded  modes,  and  can  provide  backup  capability  for 
interfacing  systems.  Nearly  30  positional  consoles  and  10  remote 
access  terminals  support  the  operation  of  each  regional  control 
center. 


The  embedded  software  is  configured  in  seven  CPCIs  and 
totals  nearly  6,000  modules  which  are  coded  in  Jovial  (J3).  This 
translates  to  nearly  330,000  DSLOC.  There  were  nearly  2,000 
software  changes  at  the  compilation  unit  level  during  development 
that  were  the  result  of  changes  to  the  requirements.  Of  the  to¬ 
tal  deliverable  compilation  units,  82  percent  were  affected  by 
these  changes.  The  changes  in  the  requirements  included  both 
clarifications  and  enhancements.  Between  the  time  at  which  the 
software  was  placed  under  configuration  control  and  project  week 
192,  approximately  6,000  actual  errors  had  been  detected. 

2 . 2  Development  Process  for  JSS  Software 

The  development  was  performed  in  two  major  phases: 
design  verification  (DVP)  and  implementation  (IP).  At  the  peak 
of  IP  over  100  persons  worked  on  software  development.  Approxi¬ 
mately  40  percent  of  the  software  was  "lifted"  from  previous  air 
defense  projects.  Most  of  the  lifted  software  was  copied  at  the 
Computer  Program  Component  (CPC)  design  level  in  the  form  of 
structure  charts.  Design  at  the  intramodule  level  was  copied  in 


the  form  of  HIPOs.  It  should  be  noted  that  the  lifted  software 
could  contain  residual  errors.  Micro-phases  completed  within  the 
IP  were  requirements  analysis,  design,  coding,  parameter  and  as¬ 
sembly  test,  integration,  independent  test,  and  system  test.  The 
project  is  now  in  the  installation  phase  which  includes  on-site 
verification  (OSV)  testing  for  each  of  the  regional  control  cen¬ 
ters  in  the  surveillance  network.  As  of  project  week  192  three 
of  the  seven  centers  had  successfully  completed  OSV  testing  and 
were  operational. 

In  developing  JSS,  Hughes  followed  the  software  develop¬ 
ment  phases  generally  accepted  by  the  industry  and  the  Govern¬ 
ment.  Those  phases  are:  requirements  analysis,  design,  coding, 
parameter  and  assembly  testing  (also  called  unit  testing),  soft¬ 
ware  integration,  independent  testing,  system  testing,  and  in¬ 
stallation  testing.  One  exception  was  the  omission  of  parameter 
and  assembly  testing  for  those  clusters  of  modules  which  were 
lifted  from  existing  Hughes  systems. 

2.2.1  Requirements  Analysis  The  JSS  software  requirement 
specifications  were  written  by  the  lead  systems  engineering 
group,  the  Systems  Division.  Generally  the  Software  Engineering 
Division  (SED)  is  consulted  or  participates  in  the  generation  of 
these  specifications.  However,  the  first  stage  for  which  SED  is 
officially  responsible  is  analysis  of  the  specification.  This 
phase  consists  of  assessing  the  feasibility  of  implementing  the 
specifications  in  software,  determining  if  there  is  existing 
software  responsive  to  similar  requirements,  and  generating  in¬ 
dependent  test  plans  based  on  specification  requirements. 

2.2.2  Design  Hughes  employs  a  programmer  team  concept  or¬ 
ganized  by  CPC  or  in  some  cases  by  Computer  Program  Configuration 
Item  (CPCI).  Examples  of  CPCs  are  weapons,  surveillance,  data 
recording,  and  displays.  On  JSS  the  System  Exercise  Set  (SES),  a 
CPCI,  was  small  enough  to  be  developed  by  one  team.  The  team 
leader  is  completely  responsible  for  the  detailed  design,  coding, 
and  checkout  of  the  software  in  the  particular  CPC.  As  a  rule, 
modules  undergo  code  reviews  by  the  team  leader,  and  upon  suc¬ 
cessful  completion  of  the  review  are  usually  placed  under  con¬ 
figuration  control.  Most  of  the  JSS  modules  were  placed  under 
configuration  control  after  completion  of  parameter  and  assembly 
testing. 

Hughes-Fullerton  employs  an  adaptation  of  Constantine 
and  Yourdon’s  (YC,  1975)  structured  design  methodology  for  decom¬ 
position  of  the  software  design  to  the  module  level.  Intramodule 
design  is  controlled  by  SED  training  courses,  individual  project 
standards,  and  detailed  design  reviews. 

Most  JSS  development  teams  utilized  the  cross-compiler 
and  computer  system  simulation  capabilities  of  both  the  Software 
Development  System  (PDP  11/70)  and  the  Amdahl  M70  during  this 


phase.  Although  the  simulator  has  some  I/O  simulation  limita¬ 
tions,  its  use  was  productive  in  detecting  errors  early  in  the 
development  when  target  computer  time  was  saturated. 

2.2.3  Coding  Hughes  coding  standards  restrict  programming 
control  structure  to  the  five  basic  structures:  Sequence,  If- 
Then-Else,  Do  While,  Do  Until,  and  Case.  The  standards  also  con¬ 
tain  module  and  data  naming  conventions,  as  well  as  statement 
labeling  conventions.  Each  module  must  have  a  single  entry  and 
single  exit,  and  no  self-modification  of  statements  during  execu¬ 
tion  is  allowed.  Like  most  recent  air  defense  applications,  JSS 
was  coded  in  the  Jovial  high  order  language,  and  the  direct  code 
option  was  limited  to  special  timing  situations  such  as  the  on¬ 
line  performance  monitoring  function  (This  function  periodically 
checks  the  system  status,  and  cannot  compete  with  the  application 
operation  cycle) . 

2.2.4  Parameter  and  Assembly  Testing  Parameter  and  assembly 
tests  provide  for  the  testing  of  specific  modules  or  groups  of 
modules  in  preparation  for  integrating  them  into  the  system 
master  version.  These  tests  emphasize  the  internal  processing  of 
modules  and  are  performed  by  the  programmer  who  coded  the 
modules.  The  main  objective  of  parameter  and  assembly  testing  is 
to  ensure  that  the  modules  under  consideration  are  reasonably 
complete  before  further  testing  on  a  broader  scale,  and  that  each 
module  or  group  of  modules  functions  properly  in  isolation.  In¬ 
formal  test  procedures  and  reports  are  generated  by  the  program¬ 
mer  and  approved  by  the  team  leader. 

2.2.5  Software  Integration  The  software  integration  activity 
is  an  orderly  sequence  of  putting  modules  together  to  perform 
software  subsystem  functions  in  accordance  with  an  integration  or 
build  plan.  This  activity  emphasizes  interfaces  between  modules, 
and  ensures  that  modules  will  function  as  designed  in  the  latest 
system  configuration.  Some  degree  of  testing  must  be  performed 
in  integration  to  provide  confidence  that  a  complete  function 
operates  as  designed,  but  not  necessarily  that  the  entire  system 
operates  correctly.  The  activity  is  directed  by  a  software  in¬ 
tegration  coordinator,  and  the  deliverable  hardware  set  is  used. 

2.2.6  Independent  Testing  The  independent  tests  validate  that 
the  performance  specifications  are  implemented  properly.  The 
testing  is  performed  by  a  test  team  that  is  organizationally  in¬ 
dependent  of  the  development  group  that  designed  and  coded  the 
software.  Test  plans  and  detailed  test  procedures  are  written  to 
validate  each  requirement  (i.e.,  "shall"  statements)  of  the  CPCI 
functional  Part  I  specifications.  All  external  inputs  are  simu¬ 
lated,  and  the  deliverable  hardware  configuration  is  employed. 
Tests  are  sensitive  to  intermediate  processing  results,  and  can 
detect  design,  coding,  and  interface  problems. 

Some  statistics  about  the  independent  testing  activity 
exemplify  the  size  of  this  effort  on  the  JSS  Project.  There  were 


nine  software  test  engineers,  including  a  team  leader,  assigned 
to  the  independent  test  team.  A  total  of  143  test  procedures 
with  14,277  test  steps  were  generated  and  conducted  for  the  seven 
CPCIs.  The  team  expended  30,332  manhours  over  32  calendar  months 
in  performing  the  independent  test  activity.  The  distribution  of 
effort  for  detailed  activities  was:  test  plan  generation  (15%), 
test  procedure  generation  (35%),  and  test  conduct  and  analysis 
(50%). 

2.2.7  System  Testing  System  tests  are  formal  acceptance  dem¬ 
onstrations  of  hardware  and  software  elements  of  the  deliverable 
operational  configuration,  which  are  performed  in  plant.  The 
software  portion  is  demonstrated  with  the  operational  hardware 
complement  by  formal  qualification  verification  (FQV)  tests  which 
are  mutually  agreed  upon  by  the  contractor  and  customer.  On  JSS 
forty-six  FQV  test  procedures  were  run  to  exercise  the  seven 
CPCIs  as  well  as  an  overall  load  test.  The  procedures  were  writ¬ 
ten  against  the  system  specification  (Type  A),  and  also  written 
to  the  requirements  "shall"  level.  A  JSS  FQV  test  procedure 
averaged  121  pages  in  length.  The  test  procedures  were  performed 
by  a  team  of  fourteen  test  engineers  from  the  Systems  Division 
with  the  assistance  of  three  test  engineers  from  the  Software 
Engineering  Division.  The  tests  were  conducted  over  a  two-month 
period,  and  witnessed  and  approved  by  the  customer. 

2.2.8  Installation  Testing  Installation  testing  consists  of 
two  phases  -  Installation  and  Checkout,  and  On-Site  Verification 
(OSV) ,  in  that  order.  The  installation  and  checkout  activities 
pertain  to  hardware  only,  and  include  air  cooling,  power-on,  and 
voltage  checks  on  the  delivered  hardware  configuration.  The  OSV 
tests  are  one  level  higher  than  the  system  tests  in  that  they 
demonstrate  that  the  system  can  complete  a  mission  scenario. 

Both  live  and  simulated  external  inputs  are  used.  The  OSV  tests 
detect  software  errors  predominantly,  since  the  installation  and 
checkout  tests  uncover  most  of  the  hardware  faults.  Further 
operational  testing  such  as  Qualification  OT&E  is  the  respon¬ 
sibility  of  the  customer  with  the  support  of  the  developer. 

On  JSS  there  were  twenty-two  OSV  test  procedures  con¬ 
ducted  by  a  team  of  10  test  engineers  including  the  team  leader. 
Most  of  the  OSV  tests  were  performed  at  each  of  the  seven  ROCC 
sites,  however  special  tests  such  as  software  reliability, 
weather,  and  peak  load  were  performed  at  selected  sites  only. 
During  live  OSV  tests  eight  different  interceptor  types  were  ex¬ 
ercised.  JSS  OSV  tests  concentrated  on  site-to-site  interfaces 
and  system  timing  characteristics. 

2.3  Input  Data  Requirements  for  Models 
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With  only  a  few  exceptions,  the  exact  data  input  re¬ 
quirements  could  be  met  with  the  JSS  data.  A  serious  exception 
occurred  with  the  Imperfect  Debugging  Model  (Goel,  1978)  since 
the  exact  times  between  successive  software  errors  were  not 
retrievable  from  the  JSS  database.  Moreover,  it  is  also  not  pos¬ 
sible  to  make  a  perfect  determination  of  whether  or  not  an  error 
is  due  to  imperfect  debugging  (i.e.  failure  to  completely  remove 
an  error)  based  on  the  data  collected  on  a  PTR.  In  fact,  many 
maintenance  PTR’s  are  new  errors  created  in  the  process  of  remov¬ 
ing  a  previous  error  rather  than  the  recurrence  of  the  previous 
error  due  to  imperfect  debugging.  Thus,  the  indicator  variables 
in  Table  2.3.1  for  the  Imperfect  Debugging  Model  are  also  ir¬ 
retrievable.  These  considerations  will  be  discussed  in  Section 
3,  along  with  rationale  for  employing  a  different  model  in  place 
of  the  Imperfect  Debugging  model. 

Another  exception  occurred  with  the  Geometric  Poisson 
Model,  although  in  this  case  the  exception  was  minor  and  easily 
corrected.  The  requirement  that  errors  be  compiled  in  equal, 
non-overlapping  time  intervals  was  not  always  convenient. 

However,  using  all  the  assumptions  and  principles  in  Moranda 
(1975)  it  was  possible  to  modify  the  likelihood  function  in 
Moranda  (1975)  to  account  for  time  intervals  of  varying  length 
and  thus  to  fit  the  Geometric  Poisson  Model  under  these  condi¬ 
tions.  See  Section  3  for  further  discussion  of  the  Geometric 
Poisson  Model. 

The  last  exception  is  relatively  minor  in  most  cases. 

One  of  the  assumptions  of  the  Imperfect  Debugging  Model  (and 
tacitly  assumed  in  all  the  other  models)  is  that  when  an  error  is 
detected,  it  is  immediately  removed  and  the  removal  time  or  "cor¬ 
rection"  time  is  negligible.  If  this  correction  time  is  not 
negligible,  then  it  must  not  be  counted  as  part  of  the  time  in¬ 
terval  during  debugging.  In  the  JSS  database,  the  correction 
time  for  each  error  is  not  available  and  cannot  be  recovered,  and 
thus  we  have  assumed  throughout  this  report  that  it  is,  indeed, 
negligible. 

In  view  of  the  presentation  given  in  Brooks  4  Motley 
(1980)  for  the  IBM  Poisson  Model,  the  list  of  data  input  require¬ 
ments  in  Table  2.3-1  seems  simplistic,  and  must  be  explained. 

The  IBM  Poisson  Model  is  a  rather  in-depth  generalization  of  the 
Jelinski-Moranda  model  and  it  allows  the  user  to  consider  several 
groups  of  software  modules  at  once.  To  do  this,  it  takes  into 
account  the  number  of  errors  detected  in  each  module  in  each 
group,  the  fraction  of  these  errors  removed,  the  fraction  of  the 
system  which  is  under  test,  and  various  other  quantities.  In 
view  of  the  time  and  budget  constraints  for  this  effort,  and  the 
lack  of  much  of  the  supporting  data  necessary  to  fit  the  IBM  mod¬ 
el  in  its  complete  generality,  we  chose  to  fit  the  model  at  the 


module  (actually,  compilation  unit)  level  so  that  the  computa¬ 
tions  and  data  requirements  would  be  simpler.  Under  these  condi¬ 
tions,  the  input  data  requirements  reduce  to  those  listed  in 
Table  2.3.1. 

In  surveying  the  references  listed  in  Table  2.3.1*  it 
was  surprising  that  no  author  was  explicit  in  specifying  the  man¬ 
ner  in  which  time  was  to  be  measured,  or  whether  the  models 
should  be  applied  during  particular  software  test  phases  (e.g., 
integration  test,  system  test,  independent  test,  etc.).  Due  to 
varying  manpower  and  the  gradual  build-up  of  software  on  a  typi¬ 
cal  software  project,  it  is  doubtful  that  calendar  time  is  an 
adequate  time  scale.  Brooks  &  Motley  (1980)  accounted  for  soft¬ 
ware  build-up  in  their  IBM  Poisson  Model,  but  no  other  author 
recognized  these  potential  problems  with  the  time  scale.  The  JSS 
database  was  carefully  monitored  to  ensure  that  there  were  no 
anomolies  in  recording  the  resolve  dates  for  the  PTR’s,  and  the 
supplemental  database  (See  Vol.  II)  contains  information  describ¬ 
ing  the  rate  of  software  build-up.  However,  there  was  no  way  to 
control  and  monitor  manpower  loading,  and  calendar  time  was  the 
only  time  measure  available  on  JSS.  While  execution  time  is  per¬ 
haps  a  better  time  measure,  schedule  and  budget  constraints  on 
JSS  would  not  allow  for  its  procurement. 

In  terms  of  the  question  of  test  phase  in  relation  to 
fitting  the  models,  there  was  prior  evidence  that  this  could  be 
am  important  factor.  In  particular,  Goel  (1980,  p.  43)  found  it 
necessary  to  eliminate  data  from  the  first  nine  out  of  ten  weeks 
of  the  validation  phase  of  formal  testing  in  his  database  in  or¬ 
der  to  obtain  the  decreasing  trend  in  his  data  necessary  to 
achieve  a  good  fit  in  the  Nonhomogeneous  Poisson  Model.  This 
would  tend  to  support  the  conjecture  that  the  models  may  be  test 
phase  sensitive.  For  this  reason,  the  additional  data  item  "test 
phase"  should  be  tacitly  assumed  as  an  additional  data  input  re¬ 
quirement  for  each  model.  Of  course,  test  phase  was  routinely 
collected  in  relation  to  each  PTR  in  the  JSS  database. 
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Input  Data  Requirements  for  Software 
Reliability  Models 

References  Input  Data  Requirements 


Moranda,1975  o  The  numbers  of  software 

errors  in  successive, 
non-overlapping  equal 
calendar  time  periods 


Goel  &  o  The  numbers  of  software 

Okumoto, 1980  errors  in  successive,  non¬ 
overlapping  calendar  time 
periods,  and  the  lengths  of 
time  periods,  or  the  calen¬ 
dar  times  between  successive 
software  errors. 


Goel, 1978  o  Calendar  times  t1#  t2,  . .., 

tn  between  successive  soft¬ 
ware  errors. 


o  Indicator  variables  yi,  y2, 
....  yn  with  y^  *  1  if  the 
ith  error  is  caused  by 
imperfect  debugging,  yA  »  0 
otherwise . 


o  The  calendar  time  to  remove 
each  error  if  this  time  is 
not  negligible. 


Schafer, et .al.  o  The  numbers  of  software 
1979  errors  in  successive  non¬ 

overlapping  calendar  time 
periods,  and  the  lengths 
of  the  time  periods. 


o  The  numbers  of  software 
errors  removed  in  successive 
non-overlapping  calendar 
time  periods. 


Binomial 


Schafer, et .al.  o  The  numbers  of  software 

1979  errors  in  successive,  non¬ 
overlapping  calendar  time 
periods,  and  the  lengths 
of  the  time  periods. 

IBM  Poisson  Brooks  &  o  The  numbers  of  software 

Motley,  errors  in  successive  non- 

1980  overlapping  test  time  inter¬ 
vals,  and  the  length  of  each 
time  interval. 


Guidelines  for  Software  Error  Data  Collection 


A  comprehensive  survey  study  of  the  experience  in  the 
collection  of  data  related  to  the  software  development  process 
(Thibodeau,  1979)  reports  that  the  same  mistakes  in  the  data  col¬ 
lection  process  have  been  repeated  over  and  over  again.  The  re¬ 
port  concluded  that  if  we  intend  to  develop  quantitative 
relationships  between  software  errors  and  their  causes  then  we 
need  to  develop  1)  automated  data  collection  techniques,  2)  con¬ 
sistent  definitions,  and  3)  a  more  manageable  error  classifica¬ 
tion  system.  This  section  contains  guidelines  for  collecting 
error  data  for  use  with  quantitative  software  reliability  models 
and  metrics.  These  guidelines  are  based  on  Hughes-Fullerton’s 
experience  in  collecting  software  error  data  from  ongoing  soft¬ 
ware  development  projects  such  as  JSS  that  use  a  semiautomatic 
data  collection  system  and  a  manageable  and  standard  error  clas¬ 
sification  scheme. 

It  should  be  understood  that  the  guidelines  presented 
here  are  for  the  collection  of  error-related  data  necessary  for 
the  direct  input  into  quantitative  software  reliability  models 
and  into  their  attendant  estimator  algorithms.  The  data  must  be 
collected  during  the  software  development  phases  in  order  for  the 
models  or  metrics  to  provide  either  immediate  assessment  or  pre¬ 
dictive  values.  As  it  will  be  seen,  data  in  addition  to  that 
required  for  direct  input  into  the  models  and  metrics  is  sugges¬ 
ted  to  be  collected  to  support  data  validation  and  to  provide 
reference  material  for  research. 

2.4.1  Approach 

The  overall  approach  to  any  error  data  collection  ac¬ 
tivity  is  to  collect  as  much  related  data  as  feasible  (without 
perturbing  important  project  milestones)  from  the  selected  soft¬ 
ware  development  project.  This  approach  allows  for  redirection 
during  the  project  to  accommodate  new  models  and  metrics  of 
interest.  Furthermore  the  collection  of  some  redundant  data  aids 
in  the  validation  process. 

A  significant  approach,  especially  with  respect  to  the 
encouragement  of  cooperation  from  the  software  projects,  is  to 
develop  a  data  collection  procedure  that  has  a  minimum  require¬ 
ment  for  participation  by  the  project.  For  example,  the  use  of 
straightforward  error  categories  and  menu-formatted  input 
requests  serve  to  minimize  the  extra  time  required  by  project 
personnel . 

2.4.2  Indoctrination 


No  one  likes  to  be  associated  with  committing  an  error. 
This  is  especially  true  in  software  where  the  manifestations  of 
the  error  may  be  catastrophic,  expensive,  or  curtail  the  progress 


of  team  members.  Consequently,  the  psychological  aspects  of 
being  responsible  for  an  error  should  be  dealt  with  early-on  in  a 
project  where  error  data  collection  will  be  performed.  It  is 
worthwhile  to  view  an  error  as  a  phenomenon  of  programming  which 
requires  study.  While  it  is  necessary  to  be  sensitive  to  pro¬ 
grammer's  reactions  when  threatened  by  exposure  of  their  errors, 
it  is  probably  healthier  to  get  the  errors  and  the  errant  out  in 
the  open  rather  than  to  cover  up  the  human  origin  of  errors.  All 
project  personnel  should  be  informed  of  the  purpose  of  the  data 
collection,  and  fully  trained  in  the  use  of  the  associated  pro¬ 
cedures  and  classifications. 

Programmers  must  be  indoctrinated  as  to  the  importance 
of  collecting  complete  and  accurate  data  for  an  upcoming  software 
engineering  study.  Every  effort  must  be  taken  to  provide  error 
classification  codes  and  definitions  at  the  programmer's  work 
stations.  If  the  codes  are  not  readily  available,  programmers 
will  tend  to  use  the  same  set  of  error  codes  for  all  situations. 
Programmers  and  test  engineers  must  also  be  reminded  that  they 
should  report  all  errors  and  not  fall  to  the  temptation  of  fixing 
a  distinguishable  error  as  a  undocumented  add-on  to  another  error 
which  is  in  work. 

2.4.3  Classification 

For  the  purpose  of  supporting  software  reliability  mod¬ 
els,  most  agree  that  a  standard  error  classification  is  preferred 
[Bowen,  1980].  It  is  mandatory  that  the  definitions  of  the  error 
data  collected  be  consistent  with  the  definitions  of  the  input 
parameters  of  the  software  reliability  models  employed.  For  ex¬ 
ample,  one  of  Goel's  models  (Goel,  1978)  includes  the  imperfect 
debugging  phenomenon.  This  error  class  must  be  clearly  defined 
as  the  incomplete  or  incorrect  correction  of  a  previously  docu¬ 
mented  error.  If  the  original  activity  was  a  requirements  change 
and  not  a  correction  then  an  associated  erroneous  fix  would  not 
qualify  as  an  imperfect  debugging  class  error. 

Existing  problem  reporting  forms  and  configuration  con¬ 
trol  systems  allow  for  entries  that  are  not  just  errors.  Other 
entry  classes  include  configuration  control  impounds,  adaptive 
changes,  updates  from  master  programs,  and  new  requirements. 

Other  obvious  extraneous  entries  are  duplicate  problem  report  and 
problem  rejection.  Accordingly  each  entry  must  be  classified  by 
at  least  a  cause  category  to  allow  selection  of  qualified  entries 
from  the  database. 

Hughes -Fullerton  has  found  that  a  minimal  set  of  two 
software  error  classifications  (Phase/Cause  and  Severity)  as  well 
as  the  erroneous  subprogram/module  are  required  to  support  the 
evaluation  of  software  reliability  quantitative  models .  Phase/ 
Cause  tells  in  which  software  development  phase  the  error  was 


introduced  and  what  the  programmer  or  analyst  did  wrong.  Severi¬ 
ty  tells  whether  the  manifestation  of  the  error  degrades  the  sys¬ 
tem  mission  performance.  The  identification  of  the  subprogram/ 
module  allows  for  reliability  assessment  to  the  functional  level. 

On  the  JSS  project  a  separate  classif ication  scheme  was 
employed  for  source  phase  and  cause.  The  following  error  causal 
classification  scheme,  which  was  tailored  from  an  RADC  scheme 
[Thayer,  1976],  was  used.  Of  the  sixteen  categories,  four  (100, 
J05,  J30,  and  J60)  did  not  quality  as  error-related. 

o  A00  —  Computational 

o  BOO  —  Logic 

o  COO  —  Data  Definition 

o  D00  —  Data  Handling 

o  E00  —  Design 

o  POO  —  Interface 

o  GOO  —  Compool  (Communications  pool) 
o  100  —  Problem  Report  Rejection 
o  J00  —  Other 
o  J05  —  Test-Only  Code 
o  J10  —  Timing  Optimization 
o  J20  —  Sizing  Optimization 
o  J30  —  Integration  of  New  Software 

o  JJjO  —  Unnecessary  Code 

o  J60  —  New  Requirements/Enhancements 

o  J90  —  Standard  Violation 


2-11 


Some  problems  were  encountered  with  consistent  interpretation  of 
the  JSS  causal  categories.  For  example,  there  was  not  a  clear 
distinction  between  Compool  (GOO)  and  Data  Definition  (COO).  In 
most  instances  GOO  was  used  for  any  change  to  the  CPCI  global 
compools,  and  COO  was  used  for  specific  error-related  changes  for 
preset  values  or  table  structures  for  local  data.  Another  con¬ 
fusing  category  was  Integration  of  New  Software  (J30).  This  cat¬ 
egory  was  intended  to  identify  the  impound  of  new  software 
modules,  however  some  programmers  used  J30  when  adding  to  an  ex¬ 
isting  module  (whether  for  correcting  an  error  or  for  implement¬ 
ing  a  new  requirement)  or  for  any  error  encountered  when  in¬ 
tegrating  software  to  software.  Fortunately  most  of  the  result¬ 
ing  inaccurate  classifications  were  obvious  when  related  PTR  data 
was  compared,  and  the  inaccuracies  were  corrected.  For  example, 
if  a  programmer  or  librarian  assigned  the  error  cause  J30,  and 
the  new  version  field  for  the  affected  module  is  not  "1.1"  then 
an  inconsistency  exists.  This  is  because  impounded  modules 
should  have  a  version  number  of  1.1  (unless  special  arrangements 
are  made  to  retain  previous  version  numbers  for  lifted  modules). 

There  are  several  methods  of  recording  an  imperfect  de¬ 
bugging  error.  An  Imperfect  Debugging  major  causal  category  can 
be  added  to  the  other  major  causal  category,  or  an  Imperfect  De¬ 
bugging  category  can  be  added  to  the  source  phase  classification. 
On  JSS,  imperfect  debugging  was  identified  by  the  source  phase 
category  of  Maintenance  (MN) .  If  separate  error  cause  and  source 
phase  classificatons  are  employed  (as  was  the  case  on  JSS),  then 
it  is  recommended  that  the  Imperfect  Debugging  category  be  added 
to  the  source  phase  classification,  because  including  it  as  a 
causal  subcategory  would  preclude  the  assignment  of  the  more 
descriptive  cause,  such  as  interface  error.  However  if  a  com¬ 
bined  phase/cause  classification  scheme  is  employed  as  suggested 
later  in  the  report,  the  needs  of  software  reliability  models  are 
adequately  supported. 

Maintenance  errors,  or  regression  errors  as  they  are 
sometimes  called,  accounted  for  only  four  percent  of  the  total 
errors  detected  through  week  192  of  the  JSS  project.  Just  con¬ 
sidering  the  Installation  phase,  the  percentage  was  twenty  per¬ 
cent.  Two  reasons  could  account  for  this  difference.  One  is 
that  a  different  configuration  control  system  was  used  during  the 
installation  phase  than  in  the  previous  phase.  The  system  em¬ 
ployed  during  installation  is  more  supportive  of  recording  multi¬ 
ple  attempts  to  resolve  a  PTR  than  the  automated  system  employed 
in  plant.  The  other  reason  is  that  the  acceptance  testing 
schedule  places  extra  pressure  on  programmers  to  resolve  errors 
quickly,  and  consequently  maintenance  errors  increase.  The  main¬ 
tenance  errors  reported  on  JSS  were  predominantely  of  the  incor¬ 
rect  solution  or  bad  patch  variety  and  few  or  none  were  of  the 
incompatible  or  ripple  effect  variety. 


In  consideration  of  the  direction  of  the  Joint  Logistics 
Commanders  [Hartwick  1979]  and  the  persistent  complaint  from  the 
programmers  assigned  to  classify  errors  that  the  existing  schemes 
have  too  many  categories,  we  recommend  the  following  combined 
source  phase/causal  error  classification  scheme  for  the  support 
of  software  reliability  modeling. 

o  REQUIREMENTS 

-  R1  Incorrect  Specification 

-  R2  Conflicting  Specification 

-  R3  Incomplete  Specification 

o  DESIGN 

-  D1  Requirements  Compliance 

-  D2  Choice  of  Algorithms 

-  D3  Sequence  of  Operations 

-  D4  Data  Definition 

-  D5  Interface 

o  CODING 

-  Cl  Requirement  or  Design  Compliance 

-  C2  Computation  Implementation 

-  C3  Sequence  of  Operations 

-  C4  Data  Definition 

-  C5  Data  Handling 

-  C6  Omitted  Logic 

-  C7  Interface 

o  MAINTENANCE 

-  Ml  Incorrect  Fix 

-  M2  Incompatible  Fix 

-  M3  Incomplete  Fix 


o 


OTHER 


(Nonreliability-related  errors) 

To  assist  error  classifiers,  we  propose  that  the  complete  clas¬ 
sification  scheme,  incuding  codes  and  brief  definitions,  be 
printed  on  the  back  of  the  hard  copy  PTR  form,  and  also  be  call¬ 
able  as  a  Help  file  during  interactive  mode  error  classification. 

2.11.4  When  To  Start 

Software  engineers  generally  agree  that  error  data  col¬ 
lection  should  start  as  early  as  possible,  in  other  words  during 
the  requirements  analysis  phase.  Unfortunately,  many  software 
developers  or  programmers  resist  error  recording  until  as  late  as 
the  integration  phase.  Most  projects  start  recording  error  data 
as  early  as  the  coding  phase  during  software  inspections  or  code 
reviews.  At  Hughes -Fullerton,  only  the  causal  classification  of 
the  error  is  recorded  during  code  reviews .  This  data  is  useful 
for  providing  immediate  feedback  for  evaluating  the  software 
development  process,  however  by  itself  it  is  not  supportive  to 
the  typical  reliability  model.  This  is  because  a  code  review  is 
a  scheduled  one-time  evaluation  and  does  not  have  progressive 
time-related  characteristics  that  are  required  by  most  models. 
Data  collected  during  the  checkout  phase  can  be  biased  by  the 
influence  of  the  individual  programmer’s  approach  to  debugging. 
Most  programmers  design  module  or  unit  tests  that  show  the  ab¬ 
sence  of  errors  rather  than  have  a  high  probability  of  detecting 
errors.  Other  variables  include  whether  or  not  a  programmer  desk 
checks  his  or  her  code  prior  to  using  static  and  dynamic  test 
analyzers . 

A  common  factor  that  influences  when  to  start  collecting 
error  data  is  the  existence  of  a  configuration  control  system. 
Most  automated  configuration  control  or  program  development 
library  schemes  control  the  access  to  modules  that  the  program¬ 
ming  staff  has  submitted  for  integration  into  the  system  under 
development.  Accordingly  each  time  a  change  is  made  to  a  module 
under  configuration  control,  records  of  the  change  are  automati¬ 
cally  generated.  In  order  to  take  advantage  of  this  automatic 
data  collection,  most  error  data  collection  starts  after  software 
is  placed  under  configuration  control. 

Since  most  reliability  models  and  metrics  are  used  in  a 
predictive  context  it  follows  that  more  accurate  results  will  be 
obtained  by  using  input  data  that  more  closely  represents  opera¬ 
tional  data.  The  operational  scenario  during  the  formal  accep¬ 
tance  test  phases  is  thought  to  be  more  representative  of  actual 
system  operation  than  the  earlier  development  phases. 

2.4.5  Procedure 

Hughes -Fullerton  has  converted  from  semiautomatic  to 
completely  automatic  configuration  control  systems  for  software 
development  projects.  Most  projects  that  started  in  1981  are 


using  the  automated  configuration  control  system  which  is  an 
integral  part  of  the  Programmers  Workbench.  This  system  contains 
a  separate  entry  for  each  Program  Trouble  Report  (PTR).  The  sys¬ 
tem  includes  special  commands  for  entering  and  changing  PTR  data 
as  the  problem  proceeds  through  each  step  such  as  the  Software 
Change  Review  Board  action,  assignment  to  group  for  resolution, 
submittal  of  resolution  to  the  Librarian,  and  verification  of  the 
resolution  after  incorporation  in  the  next  system  version.  Each 
PTR-related  command  has  an  associated  privilege  that  permits  only 
authorized  entries  or  changes  to  the  data.  Commands  are  also 
available  to  generate  summary  reports. 

Some  new  projects,  due  to  their  sensitivity  or  security 
classification  cannot  be  accommodated  by  the  automated  version  of 
the  configuration  control  system.  Other  projects  cannot  use  the 
automated  system  because  of  the  inaccessibility  of  the  system 
either  due  to  physical  location  or  cost.  Since  we  have  had  re¬ 
cent  experience  in  collecting  the  same  error  data  from  both  the 
semiautomated  and  fully  automated  systems,  some  comparisons  are 
noteworthy.  We  have  encountered  more  problems  in  the  area  of 
programmer/analyst-supplied  information  in  the  newer  automated 
system  than  in  the  earlier  systems. 

Naturally,  some  difficulties  are  to  be  expected  due  to 
the  implementation  of  a  new  system,  however,  another  influence 
is  involved.  This  influence  is  the  propensity  of  programmer/ 
analysts  to  use  short  cuts  in  the  automated  system.  A  typical 
example  of  this  situation  is  the  closure  of  a  PTR  as  a  "no 
change"  when  in  fact  there  was  a  change.  In  the  current  auto¬ 
mated  system  no-change  entries  are  not  required  to  have  the  as¬ 
sociated  classifications  and  data  entries.  The  author  of  a  pseu¬ 
do  no-change  justifies  such  a  resolution  by  noting  in  the  com¬ 
ments  field  that  the  PTR  was  resolved  under  another  PTR.  Even  in 
such  cases  where  the  reference  is  given  to  the  action  PTR,  the 
actual  resolution  data  is  overridden  inherently,  because  the  er¬ 
roneous  software  is  Identified  only  to  the  compilation  unit 
level.  Such  shortcomings  in  the  automated  system  point  to  the 
need  for  more  automated  and  human-performed  validations. 

The  JSS  data  collection  procedure  utilizes  the  con¬ 
figuration  control  system  which  is  part  of  an  automated  interac¬ 
tive  software  development  system.  The  basic  data  necessary  for 
input  to  the  software  reliability  models  (date  detected.  Cause, 
Severity,  Source  Phase,  Module  Name,  and  date  resolved)  are  auto¬ 
matically  entered  into  the  configuration  control  system  by  pro¬ 
gram  trouble  report  number,  prompted  by  a  menu  format.  The  basic 
data  which  is  recorded  by  PTR  is  shown  in  the  accompanying  PTR 
report  (Figure  2.4.1)  which  may  be  generated  for  each  PTR. 

A  special  error  qualifier  computer  program  was  developed 
that  screens  the  PTR  database  for  those  entries  that  have  causal 
categories  that  qualify  as  reliability-related.  These  qualified 
entries  are  then  consolidated  and  reformatted  to  include  only 
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those  items  of  interest  to  the  software  reliability  models  and 
metrics.  The  resulting  qualified  data  is  then  sorted  by  subpro¬ 
gram  and  date  detected,  and  then  placed  in  separate  files  for 
each  subprogram.  Each  file,  in  turn,  can  be  input  almost  direct¬ 
ly  into  the  computerized  version  of  the  reliability  model  or 
metric.  The  hard  copy  format  (see  Figure  2. *1.2)  of  the  files 
also  can  serve  as  a  historical  document  for  additional  research. 

2.M.6  Data  Validation 

Error  data  classifications  must  be  continually  validated 
for  accuracy  and  consistency.  As  much  of  the  monitoring  as  pos¬ 
sible  should  be  automated,  however  there  will  always  be  the  need 
for  human  intervention  to  monitor  those  classification  entries 
that  are  more  subjective.  Manual  monitoring  should  be  done  for 
error-prone  areas  on  a  regular  basis.  Other  data  should  be 
checked  as  required  by  sampling. 

We  have  found  that  during  the  testing  phases  —  starting 
with  integration  —  that  data  collection  cam  easily  be  preempted. 
It  is  expected  that  the  pressures  of  formal  testing  will  degrade 
the  quality  of  the  associated  data  collection.  Therefore  it  is 
recommended  that  a  quality  engineer  who  is  independent  of  the 
development  orgamization  be  assigned  to  a  project  during  formal 
testing  for  the  purpose  of  obtaining  accurate  software  error  data 
during  a  phase  when  both  time  and  tempers  can  be  short. 

When  anomalies  are  found  in  the  data  collection  pro¬ 
cedures  or  classifications,  every  effort  must  be  taken  to  have 
the  amomalies  corrected.  It  is  essential  that  these  corrections 
be  made  quickly  so  that  no  sloppy  habits  or  trends  develop.  Iso¬ 
lated  cases  involving  errors  of  minor  severity  or  requiring  the 
time  of  a  software  development  team  that  is  under  pressure  to 
meet  schedule,  usually  cam  be  discarded  without  biasing  the  model 
or  metric  results.  If  mampower  loads  permit,  a  senior  project 
person  should  be  assigned  to  troubleshoot  amd  coordinate  all  data 
collection  discrepancies  found  by  the  monitoring  amd  auditing 
activities.  In  most  cases  such  a  troubleshooter  cam  resolve 
amomalies  without  interfering  with  the  progress  of  the  project's 
development . 

2.5  Perspectives  on  Data  Collection  Costs 

One  hinderanc.  *  1  software  error  data  collec¬ 

tion,  until  recently,  has  been  the  lack  of  specification  by  cus¬ 
tomers  of  specific  quality  factors  of  interest  (e.g.,  reliabili¬ 
ty)  .  Now,  with  the  advent  of  stamdards  such  as  MIL-STD-SDS  and 
MIL-STD-SQAM  software  quality  requirements  amd  supporting  metrics 
are  being  defined.  The  FAA  Advanced  Automation  System  RFP  [FAA, 
1983]  includes  comprehensive  requirements  for  software  error  data 
collection.  In  addition  to  the  typical  MIL-S-52779A  software 
problem  reporting,  analysis,  and  corrective  action  system  plan; 
the  RFP  specifies  a  computer  program  reliability  data  collection 
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and  reporting  plan:  "This  plan  will  determine  how  computer  pro¬ 
gram  reliability  parameters  will  be  calculated,  determine  what 
information  is  necessary,  and  establish  the  methods  and  means  of 
data  acquisition."  The  definition  of  data  collection  require¬ 
ments  early  in  the  acquisition  phase  provides  a  better  under¬ 
standing  of  the  purpose  of  the  task  and  in  turn  an  accurate  al¬ 
location  of  funds  to  perform  the  task. 

Our  experience  in  performing  the  data  collection  task 
for  this  study  can  be  used  to  estimate  the  cost  of  data  collec¬ 
tion.  In  terms  of  person-hours  the  data  collection  and  monitor¬ 
ing  task  represented  0.6  percent  of  the  overall  software  develop¬ 
ment  effort.  Based  on  the  actual  person  hours  spent  on  the  data 
collection  task  of  the  RMD  study  the  cost  per  program  trouble 
report  was  0.398  person-hours  and  the  cost  per  software  module 
was  0.403  person-hours.  The  data  collection  effort  includes  the 
development  of  two  computer  programs  (an  error  qualifier  program 
and  a  report  generator  program) ,  weekly  quality  monitoring  of 
error  classifications,  and  monthly  editing  and  transmittal  of  the 
output  file.  Costs  associated  with  the  recording  of  errors 
(PTRs)  using  the  configuration  control  system  were  not  included 
in  the  data  collection  effort,  but  were  included  in  the  software 
development  effort. 

2 • 6  Distribution  Analysis  of  the  PTR  Database 

Several  distribution  analyses  of  items  of  interest 
within  the  JSS  database  were  performed  during  the  study  and  re¬ 
ported  on  a  regular  basis.  These  analyses  included  errors  per 
module  by  CPCI,  error  performance  effect  for  the  entire  project, 
phase  detected  for  the  entire  project,  error  cause  for  the  entire 
project,  and  monthly  error  rate  for  the  entire  project. 

The  analysis  results  of  errors  per  module  was  very 
interesting  because  of  the  variance  by  CPCI.  The  percentage  of 
lifted  design  is  one  factor  that  influenced  this  variance.  The 
results  listed  in  the  following  table  do  not  show  a  linear  cor¬ 
respondence  between  lifted  design  and  errors,  however.  Notice 
that  the  application  set  (APS)  has  the  highest  error  density 
(2.18),  but  lies  in  the  middle  of  the  lifted  design  range  with  a 
40  percent  value.  These  results  tend  to  indicate  that  the  more 
balanced  the  lifted  design  and  new  design,  the  more  error  prone 
is  the  resulting  software. 
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CPCI 

Errors 

Modules 

Lifted 

Design 

Errors/ 

Module 

APS 

3,462 

1,642 

40 

2.11 

OSS 

910 

682 

47 

1.33 

SES 

287 

465 

3 

0.62 

DRS 

712 

1.069 

6 

0.67 

DIS 

i*30 

1.175 

89 

0.37 

SUS 

215 

892 

82 

0.24 

The  following  table  summarizes  the  effectiveness  of  the 
various  testing  phases  on  JSS.  The  percent  of  total  errors  de¬ 
tected  values  for  both  integration  and  installation  testing  are 
high,  and  suggest  the  need  for  improvement  in  preceeding  test 
phases.  Although  there  are  some  errors  in  the  database  that  were 
detected  during  Parameter  and  Assembly  testing  (4.8%),  they  are 
not  included  because  most  unit  testing  was  performed  prior  to 
placing  modules  under  configuration  control. 


Test  Phase 


Integration 


Errors  Detected 
(%  of  Total  Testing) 

36.9 


hV 


Independent  Testing 

23.5 

System  Testing  (FQV) 

18.7 

Installation  Testing  (OSV) 

16.1 

Operation  &  Support  (QOT&E) 

0.0 

Total 

95-2 
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Of  the  entries  in  the  JSS  PTR  database  with  error  causal 
classifications  (5,232  of  the  total  6,016),  the  distribution  by 
cause  (see  following  table)  reveals  that  Logic  errors  account  for 
forty-four  percent.  This  finding  is  in  agreement  with  similar 
analyses  such  as  Glass,  1981. 


Casual  Category 

Logic 

41.7 

54.1 

52.1 

41.6 

25.4 

57.2 

43-7 

Data  handling 

17.6 

4.5 

14.2 

12.6 

33.8 

10.2 

15.9 

Interface 

10.3 

14.3 

17.0 

17.9 

12.5 

8.4 

12.8 

Design 

15.4 

8.0 

4.1 

9.1 

9.2 

10.2 

11.8 

Data  Definition 

10.7 

13.0 

8.4 

15.0 

8.8 

10.2 

11.2 

Computation 

2.7 

1.6 

3.1 

2.0 

8.1 

3.7 

2.9 

Standards 

1.1 

4.0 

0.6 

1.6 

1.8 

0.0 

1.3 

Other 

0.5 

0.5 

0.3 

0.2 

0.4 

0.0 

0.5 

Total 

100.0 

100.0 

99.8 

100.0 

100.0 

99-9 

100.0 

The  performance  effect  distribution  is  of  interest  — 
not  because  of  any  unusual  or  unexpected  results  —  but  because 
of  the  significance  of  qualifying  errors  detected  during  develop¬ 
ment  for  use  with  software  reliability  models.  It  has  been  sug¬ 
gested  that  only  the  categories  of  Critical  and  Major  are  ap¬ 
plicable  to  predicting  operational  reliability.  As  shown  in  the 
following  table  only  44  percent  of  the  database  entries  qualify 
as  Critical  or  Major. 


Performance  Effect 

Critical 

Major 

Minor 

Total 


Percent  of  Total 
6.4 
37.3 
2. 

100.0 


The  following  Figure  (2.6.1)  summarizes  the  error  detec¬ 
tion  rate  on  a  monthly  basis  for  the  JSS  software  development 
from  project  week  50  through  192.  Errors  recorded  during  the 
first  year  of  the  project  were  not  included.  Even  considering 
the  fact  that  the  data  represents  only  errors  recorded  after 
software  was  placed  under  configuration  control,  the  resulting 


skewness  to  the  right  indicates  that  errors  were  not  detected  as 
early  in  the  development  cycle  as  possible  and  as  would  be  pre¬ 
ferred.  The  spike  at  September  1981  (month  20)  represents  an 
overlap  of  the  peak  of  integration  testing  and  the  early  stage  of 
independent  testing  ,  and  the  spikes  at  September  1902  (month  31) 
and  March  1983  (month  36)  are  in  the  installation  phase.  It  is 
noteworthy  that  these  spikes  illustrate  a  nondecreasing  error 
rate  which  is  contrary  to  the  assumptions  of  most  software 
reliability  models . 


2 . 7  Description  of  JSS  Software  Configuration 

As  mentioned  earlier,  the  JSS  software  is  decomposed 
into  seven  CPCIs.  These  CPCIs  are  further  decomposed  into  CPCs 
or  functional  groups  which  are,  in  turn,  decomposed  into  compila¬ 
tion  units  which  are,  in  turn,  decomposed  into  modules.  For  the 
purposes  of  this  study,  the  System  Control  Set  (SCS)  was  combined 
with  the  Operating  System  Set  (OSS),  because  of  the  small  size  of 
the  SCS.  A  summary  of  the  hierarchy  and  size  of  the  JSS  soft¬ 
ware  configuration  is  presented  in  Table  2.7.1.  The  table  in¬ 
cludes  only  entries  that  had  qualified  errors  reported  against 
them.  In  other  words,  the  table  does  not  contain  all  of  the  com¬ 
pilation  units  in  the  JSS  software,  and  the  total  number  of 
modules  does  not  reach  the  totals  presented  earlier.  The  purpose 
of  the  table  is  to  provide  a  convenient  cross  reference  between 
compilation  units  and  groups  within  each  CPCI,  and  to  define  the 
size  of  each  compilation  unit  in  terms  of  number  of  modules. 

Note  that  compilation  unit  designators  are  not  unique  between 
CPCIs.  (For  example,  compilation  unit  kaz  in  the  Application  set 
is  not  the  same  as  compilation  unit  kaz  in  the  Operating  System 
set.)  Based  on  a  representative  sample  of  100  JSS  modules  the 
mean  module  size  is  55  executable  source  statements  and  the 
median  module  size  is  28.5  (This  finding  is  of  interest  as  it 
demonstrates  how  a  few  extremely  long  modules  can  distort  the 
mean  value,  commonly  prescribed  in  MIL-STDs,  when  used  to  monitor 
the  module  size  tendency  on  a  project). 


Table  2.7.1.  JSS  Software  Hierarchy 


Set  (CPCI) /Group  (CPC) 


Group 

(Gr) 


Compilation 
Unit  (CU) 


Nr .  of 
Modules 


Application  Set  (APS) 

Active  Correlation 


Action  Entry  Input  ab 


Console  Broadcast 


Displays 


Realtime  Control 
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Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Set  (CPCI) /Group  (CPC) 
Telling  Output 


Group 

(Gr) 

al 


Manual  Inputs 


am 


Auto  Inputs 


an 


Compilation 
Unit  (CU) 


laz 

Ibz 

lcz 

lfz 

llz 

lqz 

ltz 

lxz 


maz 
mbz 
mcz 
mez 
mf  z 
mgz 
miz 
mjz 
mkz 
miz 
mnz 
moz 
mpz 
mqz 
mrz 
msz 
mtz 
mwz 
mxz 


naz 
ncz 
nf  z 
niz 
noz 
nqz 
nrz 
nuz 
nzz 
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Nr .  of 
Modules 


1 

1 

1 

9 

1 

1 

21 

5 


1 

3 

1 

7 

9 

3 

6 

1 

5 

3 

1 
1 
1 

2 
5 
1 

11 

7 

1 


13 

16 

13 

1 

1 

1 

5 

5 

1 


IvlviV- 


-V  mV  Aa 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Set  (CPC I) /Group  (CPC) 


Group 

<Gr) 


Compilation 
Unit  (CU) 


Nr .  of 
Modules 


Repl ay 


Passive  Correlation  ap 


RTQC 


Recording 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Group 

Compilation 

Nr .  of 

Set  (CPCI)  /Group  (CPC) 

(Gr) 

Unit  (CU) 

Modules 

Action  Entries 

az 

zaz 

17 

zbz 

4 

zcz 

40 

zd  z 

49 

zez 

21 

zj  z 

1 

zkz 

25 

zl  z 

33 

zmz 

1 

znz 

1 

zoz 

1 

zqz 

1 

zr  z 

1 

zsz 

1 

zuz 

5 

zvz 

1 

zwz 

1 

Site  Adaptation 

aka 

N/A 

cea 

N/A 

cwa 

N/A 

nea 

N/A 

nwa 

N/A 

sea 

N/A 

swa 

N/A 

Com pool 

ape 

N/A 

Total 

1,203 

Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Group  Compilation  Nr.  of 

Set  (CPCI) /Group  (CPC)  (Gr)  Unit  (CU)  Modules 


Diagnostic  Set  (DIS) 


Disk  Test 

da 

dsk 

31 

Central  Computer 

Interface  Unit  Test 

db 

ciu 

5 

Interference  Test 

dc 

i  ft 

80 

Display  Console  Test 

dd 

hmd 

22 

Intercomputer  Data 

Duplexer  Test 

de 

icd 

18 

Magnetic  Tape  Test 

df 

mtu 

3 

Card  Reader  Test 

dg 

cdr 

1 

Controller  Computer 

Support 

dh 

dix 

43 

Diagnostic  Executive 

di 

cax 

4 

cex 

1 

cix 

1 

cmx 

6 

cpi 

6 

cps 

37 

cqx 

1 

max 

32 

sex 

89 

Memory  Test 

dk 

cm  a 

7 

cmb 

1 

mem 

19 

Central  Computer 
Manual  Operations 
Test 


dl 


cpm 


1 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Group 

Compilation 

Nr .  of 

Set  (CPCI) /Group  (CPC) 

(Gr) 

Unit  (CU) 

Modules 

Controller  Computer 

dm 

cpa 

24 

Processor  Test 

dpa 

3 

dpb 

17 

Remote  Access 
Terminal  Test 

do 

rat 

67 

System  Control 
Console  Terminal 

Test 

dp 

set 

23 

RMM  Function 

dq 

rmb 

1 

rmm 

14 

Line  Printer  Test 

dr 

hep 

18 

lPr 

1 

CMUX  Test 

ds 

mux 

26 

Central  Computer 

dt 

bio 

13 

Buffered  I/O  Test 

uni 

1 

Central  Computer 

du 

jax 

15 

Diagnostic  Support 

jbx 

1 

jpx 

1 

kdx 

8 

r  ix 

3 

spx 

13 

srx 

4 

tbt 

4 

t-'v 

Loaders 

dv 

ddl 

48 

fbt 

1 

m 

fdl 

25 

^  .*• 

L**  *  • 

lac 

29 

ldr 

13 

s.-' 

String  Interface 

dw 

dis 

23 

m 

Central  Computer 

Dual  Processor  Test 

dx 

dul 

25 

Compool 

dmc 

N/A 

-  .  A 


Total 


00 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Set  (CPCI) /Group  (CPC) 


Group  Compilation  Nr.  of 

(Gr)  Unit  (CU)  Modules 


gqd 

11 

grd 

54 

gsd 

8 

gtd 

67 

gud 

5 

gvd 

36 

gwd 

29 

tgd 

4 

Library/Common  rl  lad 

lbd 

lfd 

lhd 

lid 

lmd 

lqd 

ltd 

lud 

lwd 


1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


Performance  and 

Continuous  Evaluation  rp  pad 

pbd 

pcd 

pdd 

ped 

pgd 


15 

18 

12 

12 

14 

77 


Select  JRT  Data  rs 


sad 

spd 


18 

4 


Com  pool 


drc 


N/A 


Total 


Operating  System  Set  (OSS) 

Radar  Data  Processor  oa  aaz 

acz 

aos 

Card  Reader  oc  cap 


1,051 


26 

1 

1 

1 
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Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Group  Compilation  Nr.  of 

Set  (CPC I) /Group  (CPC)  (Gr)  Unit  (CU)  Modules 


OSS  Executive  oe 


CCIU 


oq 


Broadcast  Controller  oh 
ICDD  Interface  oi 


Input/Output  oj 
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eaz 
ebz 
ecz 
edz 
eez 
ef  z 
etnz 
enz 
epz 
eqz 
esz 
evz 
ewe 
ewz 
eyz 
fmz 
mbc 

gaz 

gbz 

gdz 

gep 

gjz 

glz 

gpz 

grz 

gyz 

hbz 

iaz 
iez 
i  f  z 
ipz 
i  sz 
i  tz 

dpp 

jax 

jbx 

jex 

jdx 

jgx 

jox 

jzx 


1 

1 

9 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

30 

3 

3 

1 

1 

1 

1 

1 

1 

3 

1 

1 

1 

1 

1 

1 

1 

1 

14 

1 

1 

1 

1 

1 

1 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Group 

Set  (CPC I) /Group  (CPC)  (Gr) 


Compilation 
Unit  (CU) 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Set  (CPCI) /Group  (CPC) 


Operator  input 
Processing 

Terminal  Test 

Compool 

System  Exercise  Set  (SES) 
Adaptation 

Control 

Display 

Generate  Exercise 

List 

Merge 

Noise  File 
Generation 

Library  generation 

Process  inputs 

Compool 


Group 

Compilation 

Nr .  of 

(Gr) 

Unit  (CU) 

Modules 

dbs 

17 

dgp 

1 

dip 

1 

dip 

1 

dvp 

1 

cs 

sas 

9 

ct 

tas 

6 

coc 

N/A 

mmc 

N/A 

Total 

557 

ea 

aas 

29 

ec 

CCS 

10 

ed 

das 

31 

dbs 

17 

eg 

gas 

43 

gbs 

37 

ges 

41 

el 

las 

41 

lbs 

20 

lcs 

16 

em 

mas 

53 

en 

nas 

15 

es 

sas 

14 

ev 

vas 

51 

vbs 

20 

vcs 

27 

sec 

N/A 

Total 

465 
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Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 

Group 

(Gr) 

Compilation 

Unit  (CU) 

Nr . 

of  i 

Set  (CPCI) /Group  (CPC) 

Modules 

’ 

xez 

i 

>:* 

xgz 

i 

xic 

i 

P 

xid 

2  1 

xoz 

1 

xpz 

1 

*  V. 

xtz 

i  : 

•.**■  * 

xuz 

1  j 

m 

xvz 

i 

& 

DPS  Utilization  , 

oy 

dbx 

4 

dbz 

11 

vu 

>\s, 

9 

v.-. 

Loader 

oz 

zr  z 

to 

ztp 

ztz 

6 

1 

’■y 

zup 

7 

•>; 

ldr 

13 

Safe  Data 

ol 

jad 

5 

Task  Initialization 

o2 

eah 

1 

y 

L  *  •  « 

Telling  Output 

o3 

lbz 

1 

iii 

Line  Printer 

04 

ldp 

1 

S 

lep 

lip 

1 

1 

# 

lkp 

1 

^  *  •  ■ 

■  -S 

Irp 

1 

System  Control  Set  (SCS) 

Ml 

Device  Status 

13 

Analysis 

ca 

aas 

**,  •*. 

cbs 

1 

f  ah 

1 

>j 

Central  Computer 
Input 

cc 

cas 

1 

P\ 

dap 

1 

lap 

1 

Display  Generation 

cd 

das 

2 

K‘-*\ 

dba 

9 

Lji 

dbb 

8 

F? 

dbf 

1 

pjS: 
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Table  2.7.1.  JSS  Software  Hierarchy  (Continued) 


Group 

Compilation 

Nr .  of 

Set  (CPCI) /Group  (CPC) 

(Gr) 

Unit  (CU) 

Modules 

Support  Set  (SUS) 


Adaptation 

Calculation 

sa 

adp 

28 

Symbolic  Library 

sb 

jol 

50 

Control 

sc 

con 

116 

Compool  Data 

Generator 

sd 

cdg 

10 

Symbolic  Library 

se 

csl 

57 

Symbolic  Library 

sf 

osl 

108 

Geography  Data 
Generator 

sg 

gdg 

77 

File  Operations 

am 

flo 

55 

Program  Test  Aids 

sq 

pta 

38 

Recording 

Specification 

sr 

rsg 

19 

System  Generation 

ss 

sgn 

1 

sgo 

78 

sgp 

37 

1632  Cross  Assembler 

sz 

3  Sin 

57 

Compool 

ssc 

N/A 

Total 

731 

3.  SOFTWARE  RELIABILITY  MODELS 

3 . 1  Introduction 


The  purpose  of  this  section  is  to  provide  the  necessary 
background  information  and  references  to  the  models  used  in  this 
study,  and  to  explain  what  modifications  (if  any)  have  been  made 
in  the  models  to  accommodate  the  circumstances  surrounding  the 
JSS  database.  Comments  concerning  the  model  assumptions  in  rela¬ 
tion  to  the  JSS  database  will  also  be  made  in  each  case. 

Final  subsections  have  been  included  which  contain  ob¬ 
servations  concerning  similarities  which  exist  among  the  models 
in  their  original  forms,  and  comments  on  their  applicability,  in 
general,  to  the  JSS  database.  Also  included  is  a  commentary  on 
the  methods  of  parameter  estimation  advocated  for  each  model. 

The  last  subsection  provides  rationale  for  choosing  a  measure 
derivable  from  each  model's  outputs  which  would  assist  project 
office  personnel  in  adequately  monitoring  the  formal  and 
qualification  testing  efforts  of  a  software  development  project. 

3*2  Imperfect  Debugging  Model 

The  Imperfect  Debugging  Model  was  developed  by  Goel  & 
Okumoto  (Goel,  1978;  Goel  &  Okumoto,  1978)  in  response  to  the 
need  for  a  software  reliability  model  which  would  model  the  phe¬ 
nomenon  of  uncertainty  in  error  removal/correction  in  software 
debugging.  Some  of  the  major  assumptions  of  this  model  are  list¬ 
ed  below  in  direct  quotation  from  Goel  &  Okumoto  (1978): 

"(i)  The  error  causing  a  software  failue,  when  de¬ 

tected,  is  corrected  with  probability  p(0  <_  p 
<.  1),  while  with  probability  q(p  ♦  q  =  1)  we 
fail  to  completely  remove  it.  Thus,  q  is  the 
probability  of  imperfect  debugging. 

(ii)  Errors  in  the  software  package  are  independent 
of  each  other  and  have  a  constant  occurrence 
rate . 

(iii)  The  probability  of  two  or  more  errors  occur¬ 
ring  simultaneously  is  negigible. 

(iv)  The  time  to  remove  an  error  is  considered  to 
be  negligible  in  this  model. 

(v)  No  new  errors  are  introduced  during  the  debug¬ 
ging  process. 


(vi) 


At  most  one  error  is  removed  at  correction 
time. " 


In  addition,  it  is  assumed  that  X(t),  the  number  of  errors 
remaining  in  the  system  at  time  t  (the  authors  do  not  specify  how 
t  is  measured,  i.e.,  calendar  time,  execution  time,  CPU  time, 
etc.)  is  a  semi-Markov  process  with  initial  value  X(0)=N.  The 
sojourn  times  in  each  state  are  additionally  assumed  to  be  ex¬ 
ponentially  distributed  with  failure  rate  proportional  to  the 
number  of  errors  remaining  in  the  software. 

To  aid  in  understanding  this  process  governing  the  num¬ 
ber  of  errors  remaining  in  the  software,  and  to  help  in  explain¬ 
ing  the  modifications  to  this  model  which  are  necessary  to  accom¬ 
modate  the  JSS  data,  it  is  instructive  to  describe  the  evolution 
of  the  process  in  a  qualitative  fashion.  At  time  t=0,  there  are 
N  errors  in  the  software.  After  Si  units  of  time  (where  Sj  is 
exponentially  distributed  with  failure  rate  NX  )  an  error  mani¬ 
fests  itself.  With  probability  p  (independent  of  S^)  this  error 
is  removed  instantaneously  and  with  probability  q=l-p,  the  error 
is  not  removed.  If  the  error  was  removed,  then  S2  time  units 
later  (where  S2  is  independent  of  Si ,  and  is  exponentially  dis¬ 
tributed  with  failure  rate  (N-l)X)  the  next  error  manifests  it¬ 
self,  and  is  instantaneously  removed  with  probability  p,  and  not 
removed  with  probability  q.  If  the  first  error  had  not  been 
removed,  then  S2*  time  units  later  (where  now  S2*  is  exponential¬ 
ly  distributed  with  failure  rate  NX)  the  next  error  (which  could 
possibly  be  the  first  error  again)  manifests  itself,  and  is  in¬ 
stantaneously  removed  with  probability  p,  and  not  removed  with 
probability  q.  The  process  proceeds  in  this  fashion  until  all 
errors  are  removed,  which  will  ultimately  happen  as  long  as 
0  <  p  <_  1,  and  X  >  0. 

Goel  (1978),  and  Goel  &  Okumoto  (1978)  have  provided 
methodologies  for  estimating  the  unknown  quantities  N,  X  ,  and  p 
based  on  knowing  the  successive  times  between  software  error  man¬ 
ifestations  along  with  corresponding  variables  taking  value  1  if 
the  corresponding  error  was  due  to  imperfect  debugging,  and  0 
otherwise.  In  addition,  the  authors  have  provided  a  rather  ex¬ 
haustive  analysis  of  this  model  in  terms  of  the  probability  dis¬ 
tribution  of  the  amount  of  time  to  achieve  a  specified  number  of 
errors,  distribution  of  number  of  remaining  errors,  and  expected 
number  of  errors  detected  and  remaining.  Many  of  the  expressions 
derived  in  Goel  (1978)  and  Goel  &  Okumoto  (1978)  were  subsequent¬ 
ly  simplified  in  Shanthikumar  (1980)  and  independently  in  James 
et.  al.  (1982). 

The  assumptions  quoted  previously,  and  the  method  of 
parameter  estimation  discussed  herein  deserve  comment  in  the  con¬ 
text  of  the  JSS  data.  While  assumptions  i,  iii,  and  iv  are 
plausible  for  the  JSS  data,  assumptions  ii,  v,  and  vi  probably  do 
not  hold.  Many  of  the  JSS  program  personnel  concur  that  errors 
do  not  individually  possess  a  constant  occurrence  rate,  and  they 
usually  cite  examples  relating  to  errors  existing  in  portions  of 
the  software  which  are  infrequently  exercised.  Moreover,  Nagel  & 
Skrivkan  (1982,  p.63)  offer  convincing  experimental  evidence  to 
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this  effect.  They  also  offer  evidence  which  would  tend  to  sup¬ 
port  the  assumptions  relating  to  exponential  sojourn  times,  pro¬ 
vided  that  time  is  measured  in  the  proper  unit  (e.g.  run  time, 
execution  time).  It  is  doubtful  in  view  of  the  varying  manpower, 
scheduling,  and  other  factors  that  calendar  time  would  be  an  ade¬ 
quate  measure  of  time  on  JSS. 

Finally,  the  most  serious  barrier  to  fitting  this  model 
to  the  JSS  data  is  the  unavailability  of  time -between- errors 
data.  This  data  is  unavailable  in  any  reliable  form  on  JSS  be¬ 
cause  PTRs  are  not  written  and  dated  necessarily  on  the  day  the 
error  occurred.  Moreover,  the  best  resolution  in  such  data  would 
be  to  the  nearest  day  which  would  lead  to  "ties"  in  the  input 
data  which  according  to  the  model,  are  impossible.  Another  dif¬ 
ficulty  is  that  testing  (even  at  the  module  level)  does  not  cease 
when  an  error  is  detected,  and  errors  are  not  necessarily  removed 
(or  attempts  to  remove  made)  when  the  errors  are  detected.  In 
some  cases,  an  error  is  not  removed  until  days  or  weeks  after  it 
was  detected. 

To  alleviate  some  of  these  difficulties  and  allow  some 
form  of  the  Imperfect  Debugging  Model  to  be  utilized  (however 
modified)  on  the  JSS  data,  it  is  instructive  to  consider  what 
happens  if  all  the  assumptions  of  the  Imperfect  Debugging  Model 
are  assumed  to  hold,  but  that  the  observations  are  sampled  in  a 
different  way  (l.e.,  different  than  observing  the  times  between 
each  error  manifestation  and  whether  or  not  the  error  was  due  to 
imperfect  debugging) .  In  particular,  consider  the  sequence  of 
times  between  error  manifestations  in  which  the  errors  are  actu¬ 
ally  removed .  Denote  these  times  by  T1#  T2»...TN.  Let  us  first 
consider  the  distribution  of  Tj_. 

The  first  error  manifestation  occurs  at  a  time  which  is 
exponentially  distributed  with  failure  rate  NX  .  However,  this 
is  not  equal  to  T^  unless  the  error  is  removed,  which  happens 
with  probability  p.  If  it  is  not  removed,  then  a  time  later 
(which  is  independent,  and  identically  distributed  as  the  time  to 
the  first  manifestation)  the  second  manifestation  occurs.  If  the 
bug  is  removed,  then  T^  is  the  sum  of  two  independent  and  identi¬ 
cally  distributed  exponential  random  variables  with  failure  rate 
NX  .  If  it  is  not  removed,  another  period  of  time  which  is  in¬ 
dependent  and  identically  distributed  to  the  first  two  time  peri¬ 
ods  elapses  until  the  third  manifestation.  If  the  error  is 
removed  on  this  third  try,  then  T^  is  the  sum  of  three  indepen¬ 
dent  and  identically  distributed  exponential  random  variables 
with  failure  rate  NX  .  Proceeding  in  this  fashion,  it  is  easy  to 
see  that  T^  has  the  same  distribution  as  the  random  sum 
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where  X^,  X2,...  are  independent  and  identically  distributed  ex¬ 
ponential  random  variables  with  failure  rate  NX  ,  and  L  is  a 
random  integer  valued  variable  independent  of  the  X^  and  having 
the  geometric  probability  distribution  I 

P<L=j>  *  qJ-ip;  j*l , 2,  .... 


It  is  well-known  that  such  a  geometric  convolution  of  exponential 
random  variables  is  again,  exponentially  distributed  with  failure 
rate  Nx  p.  To  see  this,  note  that  the  characteristic  function 
of  Xj  is; 

E[exp(  itXj )  ]  =  NX  /(NX  -  it) 
where  i  =  yj-l  . 


Therefore,  the  characteristic  function  of 
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which  is  the  characteristic  function  of  an  exponentially  dis¬ 
tributed  random  variable  with  failure  rate  NpX  .  That  is,  Ti  is 
exponentially  distributed  with  failure  rate  NXp.  In  general,  it 
can  be  shown  using  similar  arguments  that  T«  is  exponentially 
distributed  with  failure  rate  (N-i+l)X  p,  l£  i  £  N.  The  implica¬ 
tion  here  is  obvious:  Tj_,  To,  . . . ,  T^  constitute  the  observa¬ 
tions  in  a  Jelinski-Moranda  (Moranda,  1975 »  P-  328)  De- 
Eutrophication  Model  with  <f>  *  X  p. 


In  terms  of  fitting  the  Imperfect  Debugging  Model  to  the 
JSS  database,  it  is  true,  roughly  speaking,  that  if  only  those 
errors  which  are  actually  removed  are  considered,  then  the  Jelin¬ 
ski-Moranda  model  (or  some  variant  designed  to  handle  grouped 
data)  can  be  fit.  In  other  words,  the  two  models  are  indisting¬ 
uishable  under  this  type  of  sampling  plan.  Luckily,  Lipow  (197*0 
proposed  an  extension  of  the  Jelinski-Moranda  Model  which  was 
based  on  the  same  assumptions  as  the  original  Jelinski-Moranda 
Model,  but  which  was  based  on  grouped  input  data  as  is  available 
on  JSS.  Subsequently,  in  Schafer  et.al.  (1979>  p.3-2)  a  further 
extension  whereby  the  actual  number  of  errors  removed  at  the  end 
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of  each  time  interval  could  be  used  was  developed  for  use  on  data 
from  other  Hughes  projects.  It  is  this  version  which  is  used  in 
place  of  the  Imperfect  Debugging  Model  on  the  JSS  data. 


The  assumptions  underlying  this  model  (subsequently  re¬ 
ferred  to  as  the  Jelinski-Moranda  model)  are  slightly  less 
restrictive  than  those  of  the  Imperfect  Debugging  Model.  Assump¬ 
tion  (ii)  from  Goel  &  Okumoto  (1978)  holds,  except  due  to  the 
possibility  of  imperfect  debugging,  the  parameter  A  is  replaced 
by  0  ■  Ap.  Assumptions  (iii),  (iv),  and  (v)  also  hold.  It  is 
further  assumed  that  at  the  end  of  a  debugging-time  interval  a 
number  of  errors  are  removed  and  that  no  errors  are  removed 
during  the  time  interval.  It  is  also  assumed  that  in  an  interval 
of  time,  the  number  of  errors  detected  will  be  Poisson  dis¬ 
tributed  with  mean  proportional  to  the  number  of  errors  remaining 
in  the  software.  The  details  of  this  model  may  be  found  in 
Schafer  et.al.  (1979)-  The  initial  number  of  errors  N  ,  and  the 
parameter  0  are  estimated  by  solving  the  following  two  equations 
iteratively  for  N  and  <p  ' 


k 

i=1 


Ni 


N-M 


-  0 


i-1 


k 

I  T 
1  =  1 


=  0 


(3.2.1) 


1  k 

J _  V 

0  i  =1 


k 

1  (N-M.  ,)  t. 

i=l  1 


0 


(3.2.2) 


where  Mj  is  the  total  number  of  errors  removed  up  to  the  end  of 
the  jtn  time  interval,  r j  is  the  length  of  the  jth  time  inter¬ 
val,  Nj  is  the  number  of  errors  detected  in  time  interval  r j, 
and  k  Is  the  total  number  of  time  intervals  observed.  The  method 
of  solving  (3-2.1)  and  (3-2.2)  is  that  of  Newton-Raphson  (after 
algebraically,  eliminating  0  )  as  described  in  Appendix  B. 

In  summary  of  this  section  it  is  important  to  review 
several  key  points.  First,  we  are  not  saying  that  the  Imperfect 
Debugging  Model  is  generally  equivalent  to  the  Jelinski-Moranda 
model.  The  fact  is,  that  under  a  sampling  plan  which  only  con¬ 
siders  removed  errors,  the  statistical  analysis  of  the  two  models 
is  the  same.  Worded  differently,  under  this  type  of  sampling 
plan,  the  Imperfect  Debugging  Model  reduces  to  the  Jelinski- 
Moranda  Model. 

Secondly,  since  only  grouped  data  is  available  on  JSS, 
we  could  not  utilize  the  Imperfect  Debugging  Model  in  its 
original  form.  We  therefore  chose  to  fit  the  version  of  the 
Jelinski-Moranda  Model  developed  in  Lipow  (1971*)  and  extended  in 
Schafer  et.al.  (1979)  since  it  embodies  the  essential  assumptions 
of  the  Imperfect  Debugging  Model.  For  the  purpose  of  this  study, 
when  we  refer  to  the  Jelenski-Moranda  Model,  we  will  be  referring 
to  the  version  studied  in  Schafer,  et.al.  (1979). 
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Nonhomogeneous  Poisson  Model 
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The  Nonhomogeneous  Poisson  Model  (Goel  &  Okuiroto,  1980) 
was  a  pioneering  effort  in  the  application  of  Nonhomogeneous 
Poisson  process  modelling  to  software.  This  model  assumes  that 
the  cumulative  number  of  software  errors  detected  in  the  time 
(again,  time  units  are  unspecified  by  the  authors)  from  zero  to  t 
follows  a  Nonhomogeneous  Poisson  process  with  mean  value  function 

m(t)  =  a(l-exp(-bt))  (3-3.1) 


where  a  and  b  are  positive.  Rosner  (1965)  proposed  a  very  simi¬ 
lar  model  for  hardware  reliability  growth.  Rosner ’s  model  was 
subsequently  studied  in  Schafer,  et.al.  (1975)»  and  referred  to 
as  the  "IBM  Model".  This  model  assumed  that  the  cumulative  ex¬ 
pected  number  of  failures  up  to  time  t  was  given  by 

V(t)=  At  +  Kx  (l-exp(-K2t))  (3-3.2) 


where  A,  K^,  and  K2  are  positive  constants. 
Obviously,  (3-3.2)  reduces  to  (3.3.1)  when  A  *  0. 


The  specific  assumptions  underlying  the  Nonhomogeneous 
Poisson  Model  are  simple  and  unrestrictive  (Goel  &  Okumoto, 
1980) : 


(1)  "...  the  usage  of  the  system  is  basically 
similar  over  time." 

(2)  "...  the  number  of  failures  in  (t,  t+  At)  is 
[roughly]  proportional  to  the  number  of 
undetected  errors  at  [time]  t..." 

Assumptions  (1)  and  (2)  above  were  given  in  the  context 
of  deriving  (3-3.1)  as  a  deterministic  model  for  software  errors. 
Subsequently,  the  authors  impose  the  Nonhomogeneous  Poisson 
structure  to  account  for  random  deviations  from  (3*3.1) •  In  this 
connection,  they  also  point  out  a  third  assumption:  (3)  "...  a 
detected  error  may  not  be  removed  and  as  a  result  may  cause  addi¬ 
tional  failure(s)  at  a  later  stage.  For  the  N(t)  process,  such 
occurrences  are  counted  as  new  events." 

In  (3),  N(t)  is  the  random  cumulative  number  of  errors 
detected  by  time  t. 

Assumption  (3)  is  plausible  for  the  JSS  data,  while  as¬ 
sumption  (1)  (and  hence  assumption  (2))  are  probably  not  true 
uniformly  over  all  of  the  JSS  database.  The  evidence  of  this  is 
obvious  upon  inspecting  the  plots  of  observed  cumulative  PTR’s 
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versus  time  in  Section  3.9.  These  plots  all  show  evidence  of 
alternating  increasing  and  decreasing  error  rate  which  is  incon¬ 
sistent  with  (3.3*1)  whose  error  rate  is  given  by 


abexp(-bt) 


which  is  always  decreasing  in  time.  Possible  remedies  to  this 
situation  with  respect  to  the  JSS  database  are  to  restrict  atten¬ 
tion  to  single  Compilation  Unit  (and  thus  eliminate  the  effects 
of  software  build-up)  and  to  single  test  phases  (to  eliminate  the 
effects  of  nonhomogeneity  of  testing) .  Further  discussion  on 
this  matter  is  contained  in  Section  3.9  and  Section 

The  procedure  for  fitting  this  model  is  given  in  Goel  & 
Okumoto  (1980,  pp. 25-27).  Briefly,  the  parameters  a  and  b  are 
estimated  by  solving: 

a  ( l-exp( -btn) )  =  yn  (3-3.3) 


-bt 


at  e 
n 


n 

=  l 
i=l 


-bt. 

■  ti-l 


"bt.  1 
e  1  ]) 


-bt. 


(3.3.4) 


for  a  and  b .  In  (3*3-3)  and  ( 3 • 3 • 4 ) ,  y*  errors  have  been  de¬ 
tected  by  time  t*,  1  £  i  £  n,  and  the  Newton-Raphson  iterative 
procedure  is  used  after  algebraic  elimination  of  a  (See  Appendix 
B). 

3-4  IBM  Poisson  Model  The  IBM  Poisson  Model  is  documented  in 

Brooks  and  Motley  (1980).  It  is  a  generalization  of  the 
Jelinski-Moranda  model  in  the  following  respects: 

(1)  It  recognizes  and  attempts  to  account  for 
software  build-up  during  testing. 

(2)  It  recognizes  and  attempts  to  account  for  the 
insertion  of  errors  during  the  correction 
process. 

(3)  It  allows  data  from  several  different  groups 
of  modules  to  be  used  simultaneously  to  esti¬ 
mate  the  unknown  parameters . 

(4)  It  utilizes  grouped  data  as  is  available  on 
the  JSS  project. 

The  essence  of  the  model  remains  in  close  accord  with  the 
original  Jelinski-Moranda  model.  That  is,  the  basic  assumptions 
are  that  software  errors  are  independent  of  one  another  and  occur 


A  '• 

’  w  * 


f. 


with  equal  probability  (an  assumption  convincingly  discounted  by 
Nagel  and  Skrivan  (1982)),  and  that  the  number  of  errors  occur¬ 
ring  in  an  interval  of  testing  time  is  Poisson  distributed  with 
mean  proportional  to  the  number  of  errors  "at  risk"  at  the  begin¬ 
ning  of  the  time  interval  (this  is,  in  a  sense,  the  number  of 
errors  remaining).  As  usual,  N  initial  errors  are  assumed  pres¬ 
ent  before  testing  begins.  An  added  assumption  is  that  the  num¬ 
ber  of  errors  inserted  on  any  occasion  is  proportional  to  the 
number  of  errors  detected.  These  assumptions  are  explicitly 
given  in  Brooks  and  Motley  (1980).  Assumptions  implicit  in  their 
mathematical  derivation  are  explained  below. 

First,  the  number  of  errors  at  risk  on  test  occasion  i 
is 


Ni  =  f±  N  -  a  Qi  (3.4.1) 

where  f<  is  the  fraction  of  the  system  which  is  under  test,  is 
the  number  of  errors  detected  in  that  portion  of  the  system  under 
test  prior  to  the  ith  test  occasion.  Here,  f*  is  assumed  known, 
and  is  observed.  The  parameter  a  is  the  . ..  probability  of 
correcting  errors  in  the  system  without  reinserting  additional 
errors  and  exposing  others  to  discovery.  (Brooks  and  Motley, 
1980,  p.2-11) . "  Of  course,  like  N,  a  must  be  estimated  from 
data. 


The  second  tacit  assumption  is  that  no  errors  are 
removed  during  a  unit -test  time  interval  (only  at  the  end  of  such 
periods) . 


With  respect  to  the  JSS  data  the  IBM  Poisson  Model  is 
most  suitably  applied  at  the  CU  level  since  the  values  f*  in 
(3.4.1)  are  not  known  for  JSS.  By  considering  the  individual  CU 
as  a  system,  it  can  be  assumed  that  f*  *  1  for  all  i.  The  as¬ 
sumption  embodied  in  (3.4.1)  is  questionable  for  JSS.  In  fact, 
it  was  possible  on  JSS  to  observe  exactly  how  many  errors  had 
been  removed  prior  to  each  occasion.  This  quantity  is  related  to 
Mj  defined  in  Section  3.2,  i.e.,  M<_i  is  the  number  of  errors 
removed  prior  to  the  beginning  of  the  ith  time  interval.  Since 
this  data  is  available  on  JSS,  it  seemed  inefficient  to  estimate 
it  by  a  Q j.  as  is  done  in  (3.4.1).  Therefore,  we  considered  two 
versions  of  the  IBM  Poisson  Model.  The  first  version  has  a 
in  (3.4.1)  replaced  by  M^.^,  and  thus  eliminates  the  need  to  es¬ 
timate  a  .  The  second  version  is  exactly  the  IBM  Poisson  Model 
as  described  by  the  authors.  In  the  first  version  of  the  IBM 
Poisson  Model,  the  unknowns  N  and  <p  are  estimated  by  solving  (for 
N  and  <f> )  the  equations 


‘v 
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-  { 1  - ( 1  -4>)T  1 } 
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=  0 


(3.4.2) 
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1=1  '  L  l -( l -4>)t1  J 

where  =  N  -  M^.i,  is  the  length  of  the  test  occasion 

is  the  number  or  errors  detected  during  the  i^£  test  occasion 
and  k  is  the  total  number  of  test  occasions. 

In  the  second  version  we  considered  the  exact  model  as 
presented  in  Brooks  and  Motley  (1980,  p.2-21).  This  becomes  a 
three  parameter  model  with  the  parameters  estimated  by  solving 
the  equations  (for  <p  ,  N  and  a  ): 


k  /  N.  \ 

i-Uir-  n-(l-*)Tu).  0 


(3.4.1J) 


£  T.(l-4>)Ti'1 

i=l  1 


l-(1-<j>)Ti 


-  *1 


(3.^-5) 


i-i  (jh  nj)(^  -  ■ 


(3.^.6) 


where  ■  N  -  a  E  N . , 

i=l  J 

1-1  J 

and  z  N.  Is  defined  to  be  zero  when  1*1. 

1=1  * 

J  As  usual,  the  Newton-Raphson  iterative  procedure  was 
used  to  solve  (3. 4.2)  through  ( 3.^.6)  after  algebraically 
eliminating  N  (See  Appendix  B) . 
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3-5  Generalized  Poisson  Model  The  Generalized  Pois¬ 

son  Model  (GPM)  is  an  extension  of  the  Jelinski-Moranda  model, 
and  was  proposed  in  Schafer,  et.al.  (1979)  for  two  purposes;  to 
generalize  the  assumption  of  exponential  times  between  software 
error  detections  to  Weibull  distributions,  and  to  provide  a  model 
whose  input  data  requirements  would  match  the  data  available  on 
Hughes’  and  other  software  development  projects.  The  GPM  assumes 
that  the  number  of  errors  detected  during  the  i—  debugging  time 
interval  of  length  t ^  has  a  Poisson  distribution  with  mean 

^(N-Mi.iJr^  (3.5.1) 

where  <p  is  a  constant  of  proportionality,  N  is  the  total  number 
of  errors  present  in  the  software  initially,  and  Mj  is  the  total 
number  of  errors  removed  up  to  the  end  of  the  j^fi  debugging  time 
interval.  It  is  further  assumed  that  when  errors  are  removed, 
they  are  removed  at  the  ends  of  the  debugging- time  intervals.  In 
this  model,  like  the  previous  models,  the  authors  do  not  specify 
the  units  of  time  measurement  except  insofar  as  they  be 
"debugging- time"  units. 

In  order  to  fit  this  model  the  authors  propose  that  the 
following  three  equations  be  solved  simultaneously  for  a  ,  N, 
and  <p  based  on  k  debugging  time  intervals: 
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1=1 
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<P  I 

i=l 


(3.5.2) 
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(3.5-3) 
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l  N.  £nx,  =  <p  l  (N-M.  ,)T.a  Jinx. 
i=l  1  1  1=1  1-1  1  1 


(3-5-4) 
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The  computerized  solution  of  these  equations  is  based  on 
two-parameter  Newton-Raphson  iteration  after  algebraic  elimina¬ 
tion  of  <f> . 

In  Schafer  et.al.  (1979)  substantial  difficulties  with 
the  solution  of  (3*5-2)  -  (3 -5 .4)  along  with  generally  poor  fits 
were  reported  for  the  GPM.  However,  in  that  investigation,  the 
GPM  was  still  judged  to  fit  better  than  the  other  models  studied 
(which  included  the  Jelinski-Moranda,  Schick-Wolverton,  and  Non- 
homogeneous  Poisson  Model).  In  that  investigation,  no  effort  was 
made  to  restrict  attention  oo  single  CU  or  test  phases  and  thus 
the  effects  of  software  build-up  and  variable  test  phases  were 
included  in  the  data. 

.6  Geometric  Poisson  Model 


The  Geometric  Poisson  Model  is  presented  in  Moranda 
(1975)  in  two  versions;  one  based  on  knowing  the  times  between 
successive  software  error  detections,  and  the  other  based  on  the 
numbers  of  errors  detected  in  successive  equal  time  periods  (as 
usual,  the  author  does  not  specify  how  time  is  measured). 

For  the  JSS  data,  the  second  version  of  this  model  must 
be  considered  since  the  times  between  error  detections  are  not 
available.  In  this  second  version,  the  number  of  errors  detected 
in  the  i—  time  interval  is  Poisson  distributed  with  mean 


AK1"1,  1=1,2,... 


(3.6.1) 


where  A  >  0  and  0  <  K  <  1 . 

Note  that  (3.6.1)  is  independent  of  the  time  interval  length  as 
would  be  expected  when  all  such  intervals  are  of  the  same  length. 
On  the  JSS  database,  it  is  more  convenient  to  consider  time  in¬ 
tervals  of  different  lengths.  When  this  model  is  applied  to  time 
intervals  of  variable  length,  the  estimates  of  \  and  K  in 
(3*6.1)  become  solutions  to 


E  {  — - K  t1-l  [*  1  }  =  0 

1=1  A  L 1 - K  J 


(3.6.2) 


N.Q. ' 


-  AQ1 


(3.6.3) 


where  is  the  number  of  errors  detected  in  the  interval  of 
length  t^,  t^  =  |  ti  (tg  =0),  k  is  the  number  of  intervals, 

and  j=i 

Equations  (3-6.2)  and  (3-6.3)  result  from  differentiating  the 
log-likelihood  function 


^  log  Ai  -  Xi  -  log  (N^ !)j 


(3-6.4) 


with  respect  to  A,  and  K,  and  setting  the  resulting  equations 
equal  to  zero.  In  (3-6.4),  A*  is  the  expected  number  of  errors 
detected  in  time  interval  i  of  length  r..  To  see  how  the 
original  Geometric  Poisson  model  leads  to  this  model  for  unequal 
time  intervals,  notice  that  from  Moranda  (1975).  the  expected 
number  of  errors  in  the  first  r  units  of  time  is 


X  +  A  K  +  •  •  •  + 


xKTr'  - 


The  expected  number  of  errors  in  the  second  time  interval  of 
length  t2  is 


akti  +  akti+1  +  •••  +  akW1 


K?] 


f.  . 
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In  general,  the  expected  number  of  errors  in  the  i—  time  inter¬ 


val  of  length  t  is 


AKVl  +  AKti-l+1  +  ...  +  aK 


Vl  +  V1  =  AkS'-I  p~^Ti  j 


Equations  (3.6.2)  and  (3.6.3)  reduce  to  equations  (11)  and  (12) 
in  Moranda  (1975)  when  *  1,  1  <_  i  £  k  (with  obvious  changes 
in  notation) . 


Equations  (3.6.2)  and  (3.6.3)  are  solved  for  A  and  K 
by  algebraically  eliminating  A  and  then  applying  Newton-Raphson 
iteration  (See  Appendix  B). 


For  purposes  of  comparison  of  the  models,  it  is  neces¬ 
sary  to  derive  an  expression  for  the  total  expected  number  of 
errors  present  initially  under  this  model  (analogous  to  N  in  the 
IBM  Poisson,  GPM,  Binomial,  Imperfect  Debugging,  or  "a"  in  the 
Nonhomogeneous  Poisson  Model).  This  quantity  is  easily  derived 
from  this  model  by  noting  that  the  expected  number  of  errors  de¬ 
tectable  in  an  infinite  number  of  unit  time  intervals  is 


£  AK1'1  -  JL  . 
1-1  1-K 


In  the  context  of  the  Geometric  Poisson  Model  we  will  define 


N  *  A/ (1-K) 

for  purposes  of  comparison  to  the  other  models. 

3 . 7  Binomial  Model 

The  Binomial  Model,  like  the  GPM,  was  introduced  in 
Schafer,  et.al.  (1979).  This  model  is  based  on  observing  succes¬ 
sive  numbers  of  errors  in  successive  time  intervals  (time  units 
not  specified).  The  distributional  assumptions  are  that  the  con¬ 
ditional  distribution  of  given  N^,  N2.  ...»  is  binomial 

with  parameters  *_! 


m 


and  pj  =  l-e_aTj  where  N  is  the  number  of  initial  errors,  and 
isJthe  number  of  errors  detected  in  the  time  interval  of 
length  r^.  Thus,  the  expected  number  of  errors  detected  in  time 
interval  is 


(N-V  N  )(l-e"aTi) 
j=1  J 


(3-7.1) 


where,  as  usual,  the  summation  in  (3-7.1)  is  defined  to  be  zero 
when  i=l. 

In  Schafer  et.al.  (1979),  it  was  recommended  that  the  estimates 
of  N  and  a  be  obtained  using  a  least  squares  technique,  i.e.,  by 
minimizing 

k  r  i-1  1  2 

IN.-  (l-e"aTi )(N-I  N.) 

1=1  L  1  '  j=i  3  J 


This  led  the  authors  to  define  the  estimates  of  a  and  N  to  be 
solutions  to 


k  k  2 

l  N.(l-eaTi)  =  I  (N-M.  1)(l-e-aTi) 
i=l  1  i=l  1-1 


(3.7.2) 


k 

I  N  t  (N-M.  , )e’aT1 
i=l  1  1  1-1 


k 

1=lTi(N'M1-1)2(1-e"aTl)e"aTi  (3.7.3) 


Equations  (3.7.2)  and  (3-7.3)  are  solved  iteratively 
using  Newton- Raphs on  techniques  (See  Appendix  B) . 
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Similarities  Among  the  Models 


There  are  some  striking  similarities  among  the  software 
models  under  investigation  in  this  study.  In  fact,  they  all  pos¬ 
sess  striking  similarity  to  the  Jelinski-Moranda  model,  in  one 
form  or  another.  For  example,  in  Section  3.2,  it  was  shown  that 
under  a  sampling  plan  which  considers  only  removed  errors,  the 
Imperfect  Debugging  Model  actually  reduces  to  the  original 
Jelinski-Moranda  De-Eutrophication  Model. 

The  Geometric  Poisson  Model  appears,  on  the  surface,  to 
possess  many  different  qualities.  However,  the  models  are  really 
quite  similar  as  the  following  argument  will  show.  Under  the 
extension  of  the  Jelinski-Moranda  model  proposed  by  Lipow  (1971*) 
(and  derived  entirely  within  the  assumptions  of  the  Jelinski- 
Moranda  model  except  for  the  added  assumption  that  the  errors  are 
removed  at  the  end  of  each  time  period)  the  expected  numbers  of 
errors  to  occur  in  the  first,  second,  third,  etc.,  debugging  time 
intervals  are 


1st  interval:  0N 

2nd  interval:  ^>(N-N^)T2 

3rd  interval:  ^(N-N^  -  N2)  Tj 

(3.8.1) 


i-1 

i—  interval:  #(N-  S  '  N.)  r 

j  i 

j=l 

and  so  on,  where  Nj  is  the  number  of  errors  detected  during  the 
iill  interval.  The  first  step  to  reduce  this  scenario  to  that  of 
the  Geometric  Poisson  is  to  assume  (for  the  moment)  that  r««l, 
for  all  i.  Strictly  speaking,  the  values  in  3-8.1  are  condition¬ 
al  expectations :  conditioned  on  the  past  observations.  The 
Geometric  Poisson  Model  simply  uses  the  unconditional  expecta¬ 
tions  of  the  expressions  in  (3.8.1)  to  account  for  the  possibili¬ 
ty  that  not  all  of  (or  possibly  more  than)  the  errors  observed 
in  a  time  interval  are  actually  removed.  Thus,  for  the  Geometric 
Poisson  Model,  the  expected  (unconditional)  number  of  errors  to 
occur  in  the  second  interval  is 

(N  -  0N)  =  N  (1-0) ; 


the  expected  number  to  occur  in  the  third  interval  is 


(N  -  0N  -  0N(l-0))  =  N(l-0)*; 

and  so  on.  The  expected  (unconditional)  number  of  errors  in  the 
iiG  interval  is  simply 

(0N)(l-0)i-1 

which  is  exactly  the  Geometric  Poisson  Mean  value  function  with 
(see  Section  3.6)  X  *  0N  and  K  *  1-0. 

The  Nonhomogeneous  Poisson  model  may  be  thought  of  as  a 
continuous  analog  to  the  Jelinski-Moranda  model  in  the  sense  that 
the  mean  value  functions  are  based  on  similar  principles.  That 
is,  for  the  mean  value  function  of  the  Nonhomogeneous  Poisson, 
m(t),  it  is  assumed  that  for  constants  a  >  0,  and  b  >  0, 

m(t+h)-m(t)  s  b(m(co  )-m(t))h 

i.e.,  the  expected  number  of  errors  in  the  interval  of  time  from 
t  to  t+h  is  proportional  (approximately,  for  small  h)  to  the  num¬ 
ber  of  error  remaining  (expected)  at  time  t.  This  is  exactly  the 
Jelinski-Moranda  "principle".  An  even  more  striking  resemblance 
occurs  with  the  Geometric  Poisson  Model.  Assuming  for  the  moment 
that  all  time  intervals  are  of  width  one  unit,  the  expected  num¬ 
ber  of  errors  in  interval  i  is,  according  to  the  Geometric  Pois¬ 
son  Model, 

XK1’"1  (3-8.2) 


and,  for  the  Nonhomogeneous  Poisson  Model, 


a(exp( -b(i-l) )-exp( -bi) ) 


(3.8.3) 


But,  notice  that 


which  is  exactly  (3-8.2)  with 
A=  aeb/(eb-l )  and  K=e"b. 

Going  one  step  further,  in  tie  Geometric  Poisson  Model 
with  unequal  time  intervals  of  integer  length,  the  expected  num¬ 
ber  of  errors  in  the  i££  time  interval  of  length  Ti  time  units 


AN 


AN 


•  •  +  AK 


(3.8.4) 


where  tj  =  £j*lTj  (to  =  0).  For  the  Nonhomogeneous  Pois 
son  Model,  the  expected  number  of  errors  to  occur  in  the  iin  in¬ 
terval  of  length  Ti  *  t^  -  t^j, 

is 


i) 


(3.8.5) 


Notice  that  (3.8.4)  may  be  rewritten  as 


/  A  \  /  i  -1 

(  Tq<  )(e 


UnK  tjtnK 
-e  1  ) 


which  is  exactly  (3.8.5)  with  a  ■  A/(i-r),  and  b  *  -In  K,  or 
solving  for  A  and  K,  A*  aeb/(eb-l)  and  K  =  e“b,  just  as  in  the 
case  of  equal  time  intervals.  We  ja ay  thus  conjecture  now  that 
the  Geometric  Poisson  Model  and  the  Nonhomogeneous  Poisson  Model 
will  give  the  same  results  for  a  and  b  (or  A  and  K)  under  the 
reparametrization 


a  *  A/( 1-K) 

b  -  -In  K 


(3.8.6) 


when  the  time  intervals  are  of  integer  length. 


The  similarities  between  the  IBM  Poisson  and  the  Jelin- 
ski-Moranda  Models  are  spelled  out  in  Brooks  &  Motley  (1980), 
while  the  similarities  between  the  GPM  and  the  Jelinski-Moranda 
model  are  pointed  out  in  Schafer  et.al.  (1979). 


The  Binomial  Model  is  also  seen  to  be  similar  to  the 
Jelinski-Moranda  model  by  comparing  mean  value  functions.  For 
the  Binomial  Model,  the  expected  number  (conditional)  of  errors 
to  be  detected  during  the  i£±l  interval  of  length  is 


i-1 


(3.8.7) 


(N-.E  N.)(l-exp(-ax.)) 

0=1  J  1 

Using  the  approximation  l-exp(-h)  a  h  when  h  is  close  to  zero, 
(3-8.7)  can  be  approximated  by  1, 


which  is  exactly  given  in  (3-8.1)  (when0=a)  for  the  Jelinski- 
Moranda  Model.  Moreover,  when  ati  is  small  for  each  i,  and  N 
very  large,  the  binomial  distribution  is  approximated  by  the 
Poisson  for  each  time  interval,  completing  the  analogy  between 
the  two  models. 

To  summarize,  it  may  be  conjectured  that  when  all  models 
are  seen  to  fit  a  particular  software  error  dataset,  the  esti¬ 
mates  of  the  initial  number  of  errors  (or  the  analogous  quantity) 
should  be  very  close,  as  should  be  the  estimates  of  the  "<f>" 
parameter  except  for  the  parameter  <j>  in  the  Generalized  Poission 
Model.  To  facilitate  these  comparisons  in  subsequent  analyses, 
the  model  parameters  in  each  case  can  be  translated  into  N  and<£ 
according  to  Table  3-8.1. 


Table  3-8.1 


Parameter  Translations  Relating  to  the 
Jelinski-Moranda  Model 


Mnrfri /  Formula  For:  Comment, 

Geometric  Poisson 

X  ,  K 

X/ ( 1-K) 

1-K 

X/(l-K)  is 
the  expected 
number  of 
errors  in  in¬ 
finite  time 

Jel insk i -Moranda 

N  ,<t> 

N 

$ 

Nonhomogeneous  Poisson 
a,  b 

-  ... - 

a 

b 

a  is  the  ex¬ 
pected  number 
of  errors  in 
infinite  time. 

Generalized  Poisson 

N,  <t>  ,a 

N 

■ 

This  value  of 
<f>  will  not  be 
comparable  to 
the  other 
models  due  to 
the  parameter  a 

IBM  Poisson 

N.  <t>  .  a 

N 

Binomial 

N,  a  N 

a 

Applicability  of  the  Models  to  the  JSS  Database 


There  is  no  way  to  absolutely  prove  or  disprove  the 
validity  of  all  the  software  reliability  models'  assumptions  in 
relation  to  the  JSS  database.  Some  of  the  assumptions  are  ob¬ 
viously  not  satisfied  (e.g.  that  no  new  errors  are  introduced 
during  the  debugging  process,  or  that  at  most  one  error  is 
removed  at  correction  time),  while  others  are  plausible,  but  not 
verifiable  absolutely  (e.g.  that  the  times  between  software  error 
occurrences  are  exponentially  distributed,  or  that  the  number  of 
errors  in  a  fixed  interval  of  time  is  Poisson  distributed) . 

Other  assumptions  have  been  shown  "statistically"  to  be  untrue 
for  software  in  general  (e.g.  that  all  errors  have  the  same  con¬ 
stant  rate  of  occurrence;  see  Nagel  and  Skrivan  (1982)).  More¬ 
over,  with  the  variability  of  total  manpower  effort,  the  software 
build-up  during  testing,  and  the  variability  of  test  phase,  it 
should  not  be  expected  that  any  of  the  models  would  fit  the  JSS 
database  over  all.  To  illustrate  the  reason  for  this.  Figures 
3.9.1  through  3.9.12  show  the  cumulative  number  of  errors  de¬ 
tected  for  the  six  CPCIs,  and  the  cumulative  number  of  errors 
removed  for  the  six  CPCIs  as  a  function  of  (calendar)  time.  The 
effects  of  variability  in  manpower  effort,  software  build-up,  and 
test  phase  are  possibly  manifested  as  changes  in  inflection  in 
these  plots.  All  of  the  software  reliability  models  would  pre¬ 
dict  a  cumulative  error  detected  curve  which  is  concave  downward 
everywhere,  i.e.  a  decreasing  error  rate  everywhere.  The  curves 
in  Figures  3-9.1  through  3. 9- 12  show  regions  (often  more  than 
one)  of  increasing  error  rate. 

At  this  point  it  is  tempting  to  conclude  that  the  models 
do  not  fit  the  JSS  database.  Indeed,  not  much  attention  has  been 
given  to  such  details  as  varying  manpower  effort,  test  phase,  and 
software  build-up  (except  for  Brooks  and  Motley  (1980)).  As  men¬ 
tioned  earlier,  Goel  (1980,  p.43)  apparently  observed  the  phe¬ 
nomenon  of  a  transient  increasing  error  rate  in  his  database  when 
fitting  the  Nonhomogeneous  Poisson  Model  and  found  it  necessary 
to  censor  his  first  nine  weeks  of  data  because,  as  he  states, 
they  were  "...  interested  in  analyzing  the  software  failures  over 
the  period  when  they  are  decreasing."  As  seen  in  Figures  3.9.1 
through  3-9.6,  there  can  be  more  than  one  distinct  period  of 
software  error  rate  decrease,  in  general. 

It  is  conceivable  (even  obvious  in  some  cases)  that  test 
phase  transitions  can  cause  changes  in  the  rate  of  error  detec¬ 
tion.  It  is  also  obvious  that  software  build-up  also  effects  the 
rate  of  error  detection.  Therefore,  in  order  to  apply  the  soft¬ 
ware  reliability  models  to  the  JSS  database  with  any  degree  of 
success,  it  is  necessary  to  apply  the  models  to  a  given  compila¬ 
tion  unit  (CU)  and  a  given  test  phase.  By  restricting  attention 
to  a  given  CU,  the  software  build-up  problem  is  reduced  since  all 
the  software  in  a  CU  enters  configuration  control  at  once.  Also, 
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by  restricting  attention  to  a  single  test  phase  the  testing  in¬ 
tensity  and  methodology  remain  fairly  consistent. 


The  effects  of  varying  manpower  on  JSS  could  not  be  con¬ 
trolled,  although  we  suspect  that  the  major  effects  were  test 
phase  and  software  build-up,  with  manpower  variation  playing  a 
lesser  role. 

3 • 10  Parameter  Estimation 


In  this  investigation  we  have  used  the  equations /methods 
of  parameter  estimation  advocated  by  the  respective  software 
reliability  model  authors.  These  methods  are  described  as 
follows . 


Based  on  a  set  of  observations  nj_,  n£*  ...  n^  of  the 
number  of  errors  observed  in  k  successive,  nonoverlapping  time 
periods  of  lengths  r t ?,  ...  Tk,  the  natural  logarithm  of 
the  models  likelihood  function  evaluated  at  the  observations  is 
differentiated  with  respect  to  each  parameter,  and  the  resulting 
expressions  set  equal  to  zero  and  solved  for  the  parameter 
values.  This  method  of  estimation  has  been  referred  to  by  the 
authors  as  "maximum  likelihood",  but  this  is  not,  strictly  speak¬ 
ing,  the  maximum  likelihood  technique  for  the  following  reasons. 

First,  all  but  two  of  the  models  have  the  parameter  N 
(the  number  of  initial  errors)  which  is  known  to  be  positive  in¬ 
teger  valued  and  at  least  as  large  as  the  number  of  errors 
removed  from  the  software  under  investigation.  Hence,  differen¬ 
tiation  with  respect  to  N  is  not  appropriate,  and  moreover,  the 
solutions  to  the  resulting  equations  do  not  yield  integer  values 
for  N.  Secondly,  other  parameters  also  have  restrictions  (a  >  0, 
b  >  0  in  the  Nonhomogeneous  Poisson  Model; 

0<K<1,  X  >  0  in  the  Geometric  Poisson  Model ;  <P  >  0  in  the 
Generalized  Poisson  Model,  Binomial  Model,  IBM  Poisson  Model,  and 
Imperfect  Debugging  Model).  This  estimation  technique  advocated 
by  the  authors  does  not  guarantee  that  these  restrictions  will  be 
met.  Finally,  sotting  the  derivatives  of  the  log  of  the  likeli¬ 
hood  function  equal  to  zero  and  solving  for  the  parameter  values 
does  not  necessarily  yield  parameter  values  which  indeed  maximize 
the  likelihood  function  (which  is  the  principle  of  the  maximum 
likelihood  technique).  For  these  reasons,  we  will  refer  to  this 
method  as  "pseudo  maximum  likelihood  estimation". 

In  the  case  of  the  Binomial  model,  a  different  technique 
is  used.  In  that  model  the  authors  suggest  that  the  parameters  N 
and  a  be  estimated  by  solving 

9  a  u 

where 

k  i-1 

Q  *  E  <Ni  -  ( l-e“aTl  XN-Z  N.)>2. 


1 


This  technique  must  be  termed  "pseudo  least  squares"  for  the  same 
or  similar  reasons  cited  above  for  the  pseudo  maximum  likelihood 
case.  That  is,  this  technique  does  not  necessarily  result  in 
minimizing  Q,  and  it  does  not  yield  estimates  for  N  and  a  which 
are  integer  valued  and  positive,  respectively. 

To  our  knowledge,  it  is  not  known  under  what  conditions 
these  estimators  possess  good  statistical  properties  (i.e.  con¬ 
sistency,  efficiency,  asymptotic  normality,  etc.).  In  a  simula¬ 
tion  study  performed  in  Schafer  et.al.  (1979),  it  was  observed 
that  these  techniques  apparently  do  lead  to  consistent  (i.e. 
stochastically  convergent  to  the  true  parameter  value)  and 
asymptotically  normal  estimators  for  the  Jelinski-Moranda, 
Generalized  Poisson,  Binomial,  and  Nonhomogeneous  Poisson  models. 
Moreover,  the  same  simulation  study  showed  that  the  variance  of 
the  pseudo  maximum  likelihood  estimators  was  less  (both 
asymptotically  and  as  observed  in  the  small  sample  simulations) 
than  the  corresponding  psuedo  least  squares  estimators  for  the 
Jelinski-Moranda  and  Nonhomogeneous  Poisson  Models. 

While  these  results  suggest  that  the  pseudo  maximum 
likelihood  and  pseudo  least  squares  estimators  advocated  by  the 
authors  of  the  models  do  yield  "good"  estimates  when  the  model 
assumptions  are  satisfied,  it  was  also  noted  in  Schafer  et.al. 
(1979)  that  the  lack  of  starting  points  for  the  Newton-Raphson 
procedure  for  solving  the  equations  defining  the  estimators  was  a 
serious  problem.  That  problem  prevailed  in  this  study  also. 

Thus,  often  in  the  course  of  estimating  the  model  parameters  the 
iterative  Newton-Raphson  procedure  did  not  converge,  or  it  con¬ 
verged  to  parameter  values  which  were  not  allowable  (e.g.  nega¬ 
tive  values  for  N) .  These  difficulties  are  very  damaging  to  the 
models  in  terms  of  their  use  by  software  acquisition  managers. 


3.11  A  Software  Reliability  Measure  Derivable  from  each  of 
the  Models 


As  required  by  the  statement  of  work,  it  was  necessary 
to  derive  measures  from  each  software  reliability  model's  outputs 
which  would  provide  assistance  to  project  office  personnel  in 
monitoring  formal  and  qualification  testing  of  software  projects. 
By  "model  outputs"  is  meant  the  parameter  estimates  which  result 
from  fitting  the  model.  These  parameters  and  their  physical 
interpretations,  are  described  in  Sections  3.2  through  3.7. 


In  surveying  the  various  reliability  measures  proposed 
by  the  authors  of  the  models,  we  found  most  of  them  to  be  inade¬ 
quate  because  that  they  were  time-unit  dependent.  Because  only 
calendar  time  is  collected  in  the  JSS  database,  and  because  this 
time  measure  is  not  uniformly  representative  of  test  phase  time 
nor  system  operational  time,  we  do  not  recommend  such  measure^,  as 
mean  time  to  next  error,  mean  time  to  achieve  a  certain  number  of 
remaining  errors,  probability  distribution  of  the  time  to  next 
error,  etc.  These  measures  can  be  very  misleading  to  project 


personnel  when  predicted  based  on  models'  fit  to  calendar-time 
data. 


The  measure  we  feel  would  be  of  most  assistance  to  proj¬ 
ect  personnel  is  the  number  of  errors  remaining.  This  measure 
has  many  advantages,  the  most  important  of  which  is  the  fact  that 
the  purpose  of  software  testing  is  to  identify  (and,  as  a  result, 
remove)  software  errors.  Thus,  residual  errors  is  a  direct  mea¬ 
sure  of  the  effectiveness  of  the  debugging  process.  Another  ad¬ 
vantage  of  this  measure  is  that  it  can  be  directly  computed  from 
the  outputs  of  each  model,  greatly  facilitating  comparison  (actu¬ 
ally,  for  the  Nonhomogeneous  Poisson  and  Geometric  Poisson  Mod¬ 
els,  the  analogous  measure  is  the  expected  number  of  residual 
errors).  Another  strong  advantage  is  that  this  measure  is  time 
independent . 

Table  3*H-1  lists  the  formulas  for  estimating  residual 
errors  for  each  model  in  terms  of  the  model  parameter  estimates. 


Table  3. 11.1 

Formulas  for  Residual  Errors 


Model 


Residual  Errors  Estimate* 


Modified  Imperfect 
Debugging 

Nonhomogeneous  Poisson** 
Geometric  Poisson** 


A 


N  -  M 


a  -  M 

X/(l-K)  -  M 


IBM  Poisson  N  -  M 

A 

Generalized  Poisson  N  -  M 

Binomial _ N  -  M 

*  M  is  the  total  number  of  errors  removed,  "hats" 
signify  estimates. 

**  These  are  actually  estimated  expected  residual  errors 


4.  RESULTS  OF  SOFTWARE  RELIABILITY  MODEL  FITTING 

4 . 1  Data  Compilation 

The  most  important  investigation  of  this  effort  was  the 
actual  application  of  the  software  reliability  models  to  the  JSS 
data  to  determine  their  degree  of  applicability.  To  accomplish 
this,  it  was  first  necessary  to  compile  a  number  of  datasets  from 
the  JSS  database  on  which  to  apply  the  models.  The  rationale  for 
choosing  these  datasets  was  established  to  insure  that,  as  far  as 
possible,  none  of  the  models'  assumptions  were  violated  because 
of  the  way  in  which  the  JSS  data  were  compiled. 

This  rationale  included  the  selection  of  data  from  four 
test  phases  only;  IT  (integration  test),  SD  (independent  test), 

ST  (System  Test),  and  IN  (installation  test).  Each  dataset  was 
segregated  by  test  phase  to  guard  against  the  effects  of  testing 
intensity  on  the  software  error  rate.  Each  dataset  was  also 
segregated  by  Compilation  Unit  (CU)  to  guard  against  the  effects 
of  software  build-up  (since  all  the  software  in  a  given  CU  enters 
the  test  phase  at  the  same  time),  and  to  provide  the  maximum  num¬ 
ber  of  errors  per  dataset  (the  single  module  error  rates  are  to 
small) . 


A  listing  of  this  data  is  given  in  Table  4.1.1.  In  this 
table,  each  dataset  is  identified  by  the  CPCI,  CU,  and  test  phase 
(e.g.  APS,  APC ,  IT).  The  first  column  of  numbers  represents  the 
successive  numbers  of  errors  detected  in  successive  intervals  of 
time.  The  second  column  gives  the  time  interval  lengths  in  cal¬ 
endar  weeks.  The  third  column  is  the  total  number  of  errors 
removed  prior  to  the  beginning  of  the  corresponding  time  period. 
The  fourth  and  fifth  columns  are  cumulative  total  time  and 
cumulative  total  errors  detected.  This  data  was  extracted  from 
the  JSS  database  as  it  existed  during  week  182  of  the  implementa¬ 
tion  phase  (IP).  Subsequently,  two  compilation  units  (AAZ,  MMC) 
were  relocated  from  CPCI  APS  to  CPCI  OSS.  Thus,  the  installation 
phase  was  not  complete.  All  data  is  from  IF. 

The  time  intervals  were  chosen  so  that  no  errors  were 
removed  during  the  time  intervals,  but  only  at  the  beginning  of 
the  time  periods.  This  was  accomplished  by  choosing  the  time 
interval  endpoints  to  be  the  times  at  which  error  removals  clus¬ 
tered  in  the  raw  data.  This  selection  of  time  intervals  was 
choosen  solely  for  the  purpose  of  attempting  to  satisfy  the  model 
assumptions . 

At  the  end  of  each  dataset  in  Table  4.1.1,  the  total 
number  of  errors  observed  and  removed  are  printed  for  the  par¬ 
ticular  dataset.  These  numbers  will  generally  be  one  more  than 
the  last  number  in  the  cumulative  error  count  column  and  error 
removal  column  for  the  dataset.  The  reason  for  this  is  that  it 
was  not  known  exactly  how  much  test  time  was  accumulated  when  the 


TABLE  4.1.1 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERRORS  TIME  CUM.  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  A PC  IT 


TOTAL  ERRORS  DETECTED-  35  TOTAL  REMOVED-  35 
DATA  BEGINS  AT  WEEK  64  OF  IMPLEMENTATION  PHASE 


APS  A PC  ST 


TOTAL  ERRORS  DETECTED-  16  TOTAL  REMOVED-  16 
DATA  BEGINS  AT  WEEK  151  OF  IMPLEMENTATION  PHASE 


TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


CPCI  CU  PHASE 


ERRORS  .  TIME  CUM.  ERRORS  CUM.  ERRORS  J 

DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  A PC  IN 


2 

1 

0 

1 

2 

1 

1 

1 

2 

3 

9 

6 

4 

8 

12 

1 

2 

11 

10 

13 

4 

2 

12 

12 

17 

3 

1 

13 

13 

20 

0 

2 

20 

15 

20 

TOTAL  ERRORS  DETECTED-  21  TOTAL  REMOVED-  21 
DATA  BEGINS  AT  WEEK  163  OF  IMPLEMENTATION  PHASE 


APS  ZEZ  IT 


0 

56 

0 

56 

0 

1 

18 

1 

74 

1 

12 

13 

2 

87 

13 

3 

2 

3 

89 

16 

3 

1 

11 

90 

19 

1 

3 

19 

93 

20 

1 

2 

21 

95 

21 

1 

4 

22 

99 

22 

1 

18 

23 

117 

23 

TOTAL  ERRORS 

DETECTED 

-  24 

TOTAL  REMOVED- 

24 

DATA  BEGINS 

AT  WEEK 

87  OF 

IMPLEMENTATION 

PHASE 

APS  ZEZ  SD 


0 

3 

0 

3 

0 

19 

6 

1 

9 

19 

1 

1 

3 

10 

20 

2 

1 

9 

11 

22 

5 

2 

10 

13 

27 

4 

1 

20 

14 

31 

20 

3 

21 

17 

51 

8 

1 

38 

18 

59 

1 

1 

39 

19 

60 

9 

3 

54 

22 

69 

0 

2 

67 

24 

69 

3 

2 

69 

26 

72 

1 

2 

72 

28 

73 

0 

3 

73 

31 

73 

.H 


3 


■i 

-i 


3 


4 


TOTAL  ERRORS  DETECTED-  74  TOTAL  REMOVED-  74 
DATA  BEGINS  AT  WEEK  114  OF  IMPLEMENTATION  PHASE 


TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERRORS  TIME  CUM.  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  ZEZ  ST 


0 

1 

0 

1 

0 

3 

3 

1 

4 

3 

5 

2 

2 

6 

8 

8 

3 

8 

9 

16 

1 

1 

14 

10 

17 

5 

8 

18 

18 

22 

2 

4 

22 

22 

24 

0 

1 

23 

23 

24 

1 

4 

24 

27 

25 

TOTAL  ERRORS 

DETECTED- 

26 

TOTAL  REMOVED- 

26 

DATA  BEGINS 

AT  WEEK  134  OF 

IMPLEMENTATION 

PHASE 

APS  ZEZ  IN 


5 

3 

0 

3 

5 

0 

2 

2 

5 

5 

1 

2 

3 

7 

6 

3 

5 

6 

12 

9 

0 

3 

9 

15 

9 

TOTAL  ERRORS  DETECTED-  10  TOTAL  REMOVED-  10 
DATA  BEGINS  AT  WEEK  1S6  OF  IMPLEMENTATION  PHASE 


APS  ASZ  IT 


3 

54 

0 

54 

3 

0 

2 

1 

56 

3 

0 

4 

2 

60 

3 

2 

5 

3 

65 

5 

1 

2 

4 

67 

6 

0 

1 

5 

68 

6 

2 

1 

6 

69 

8 

2 

1 

7 

70 

10 

10 

3 

11 

73 

20 

3 

1 

15 

74 

23 

1 

1 

20 

75 

24 

12 

5 

23 

80 

36 

3 

2 

32 

82 

39 

1 

1 

35 

83 

40 

1 

2 

39 

85 

41 

2 

2 

40 

87 

43 

2 

2 

42 

89 

45 

0 

2 

43 

91 

45 

2 

3 

46 

94 

47 

3 

3 

48 

97 

50 

2 

1 

49 

98 

52 

1 

1 

51 

99 

53 

1 

1 

52 

100 

54 

2 

6 

55 

106 

56 

1 

5 

56 

111 

57 

TOTAL  ERRORS 

DETECTED 

>  58 

TOTAL  REMOVED- 

58 

DATA  BEGINS 

AT  WEEK 

91  OF 

IMPLEMENTATION 

PHASE 
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TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOB  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERRORS  TIME  CUM.  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  MEZ  SD 


TOTAL  ERRORS  DETECTED-  10  TOTAL  REMOVED-  10 
DATA  BEGINS  AT  WEEK  124  OF  IMPLEMENTATION  PHASE 


APS  MEZ  ST 


TOTAL  ERRORS  DETECTED-  8  TOTAL  REMOVED-  8 
DATA  BEGINS  AT  WEEK  139  OF  IMPLEMENTATION  PHASE 


APS  HTZ  IT 


TOTAL  ERRORS  DETECTED-  12  TOTAL  REMOVED-  12 
DATA  BEGINS  AT  WEEK  92  OF  IMPLEMENTATION  PHASE 
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TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOB  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERRORS  TIME  CUM.  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  HTZ  SO 


APS  HTZ  ST 


7 

7 

0 

7 

7 

7 

1 

6 

S 

14 

4 

3 

12 

11 

IS 

1 

4 

16 

15 

19 

1 

6 

19 

21 

20 

0 

1 

20 

22 

20 

TOTAL  ERRORS 

DETECTED- 

21 

TOTAL  REMOVED- 

21 

DATA  BEQXNS 

AT  WEEK 

120  OF 

IMPLEMENTATION 

PHASE 

1 

1 

0 

1 

1 

1 

1 

1 

2 

2 

2 

2 

3 

4 

4 

1 

2 

S 

6 

5 

2 

3 

6 

9 

7 

5 

6 

7 

IS 

12 

2 

7 

13 

22 

14 

1 

10 

IS 

32 

15 

TOTAL  ERRORS 

DETECTED- 

16 

TOTAL  REMOVED- 

16 

DATA  BEGINS 

AT  WEEK 

134  OF 

IMPLEMENTATION 

PHASE 

APS  M1Z  IT 


2 

10 

0 

10 

2 

0 

2 

2 

12 

2 

5 

1 

3 

20 

7 

0 

1 

6 

21 

7 

4 

3 

S 

24 

11 

3 

2 

11 

26 

14 

2 

3 

13 

29 

16 

0 

4 

16 

33 

16 

1 

2 

17 

35 

17 

TOTAL  ERRORS 

DETECTED- 

IS 

TOTAL  REMOVED- 

18 

DATA  BEGINS 

AT  WEEK  101 

OF 

IMPLEMENTATION 

PHASE 

Mi 
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TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOB  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERBOBS  TIME  CUM.  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  MIZ  SD 


APS  MIZ  ST 


8 

S 

0 

5 

6 

6 

4 

7 

9 

12 

0 

1 

10 

10 

12 

3 

4 

13 

14 

15 

1 

2 

14 

16 

16 

0 

2 

15 

18 

16 

7 

S 

17 

23 

23 

0 

1 

19 

24 

23 

1 

1 

22 

25 

24 

0 

2 

24 

27 

24 

2 

5 

25 

32 

26 

1 

23 

27 

55 

27 

TOTAL  ERRORS 

DETECTED- 

28 

TOTAL  REMOVED - 

28 

DATA  BEGINS 

AT  WEEK 

115  OF 

IMPLEMENTATION 

PHASE 

4 

1 

0 

1 

4 

1 

1 

3 

2 

5 

2 

1 

5 

3 

7 

2 

2 

8 

5 

9 

2 

3 

10 

8 

11 

1 

1 

11 

9 

12 

1 

1 

12 

10 

13 

0 

4 

13 

14 

13 

2 

12 

14 

26 

15 

TOTAL  ERRORS 

DETECTED- 

16 

TOTAL  REMOVED- 

16 

DATA  BEGINS 

AT  WEEK 

135  OF 

IMPLEMENTATION 

PHASE 
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(CONTINUED) 

TWARE  RELIABILITY  MODELS 


ERRORS  TIME 

CPCI  CU  PHASE  DETECTED  INTERVAL 


CUM.  ERRORS 
REMOVED 


CUM.  ERRORS 
CUM.  TIME  DETECTED 


APS  DAZ  IT 


3 

2 

0 

2 

3 

0 

1 

1 

3 

3 

14 

6 

4 

0 

17 

0 

1 

0 

10 

17 

7 

3 

15 

13 

24 

IS 

5 

10 

18 

42 

2 

1 

34 

10 

44 

0 

1 

43 

20 

44 

5 

2 

44 

22 

40 

3 

4 

48 

26 

52 

3 

5 

53 

31 

55 

0 

1 

54 

32 

55 

0 

1 

55 

33 

55 

3 

13 

56 

46 

58 

0 

3 

58 

40 

58 

1 

4 

50 

53 

50 

0 

1 

60 

54 

50 

TOTAL  ERRORS  DETECTED-  60  TOTAL  REMOVED-  61 
DATA  BEGINS  AT  WEEK  104  OF  IMPLEMENTATION  PHASE 


APS  DAZ  SD 


3 

2 

0 

2 

3 

0 

7 

1 

0 

12 

3 

2 

0 

11 

15 

4 

2 

10 

13 

10 

4 

4 

14 

17 

23 

2 

5 

18 

22 

25 

0 

1 

21 

23 

25 

2 

1 

22 

24 

27 

0 

1 

24 

25 

27 

2 

6 

28 

31 

20 

1 

IS 

30 

40 

30 

TOTAL  ERRORS  DETECTED-  31  TOTAL  REMOVED-  31 
DATA  BEGINS  AT  WEEK  113  OF  IMPLEMENTATION  PHASE 
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OATA  SETS  FOR  APPLICATION  OF  SOFTWARE 


TABLE  4.1.1  (CONTINUED) 
- - —  R£LI^ 


BILITY  MODELS 


ERRORS  TIME  CUM.  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  DAZ  ST 


APS  DAZ  IN 


5 

2 

0 

2 

5 

0 

1 

3 

3 

5 

3 

2 

5 

5 

8 

3 

3 

7 

8 

11 

0 

1 

10 

9 

11 

6 

3 

11 

12 

17 

2 

2 

14 

14 

19 

1 

2 

18 

16 

20 

2 

1 

19 

17 

22 

3 

4 

21 

21 

25 

0 

1 

23 

22 

25 

1 

4 

24 

26 

26 

0 

4 

26 

30 

26 

TOTAL  ERRORS 

DETECTED* 

27 

TOTAL  REMOVED- 

27 

DATA  BEGINS 

AT  WEEK 

136 

1  OF 

IMPLEMENTATION 

PHASE 

2 

2 

0 

2 

2 

0 

2 

1 

4 

2 

1 

2 

3 

6 

3 

2 

2 

4 

8 

5 

3 

4 

S 

12 

8 

0 

1 

6 

13 

8 

0 

2 

8 

15 

8 

TOTAL  ERRORS 

DETECTED* 

9 

TOTAL  REMOVED- 

9 

DATA  BEGINS 

AT  WEEK 

164  OF 

IMPLEMENTATION 

PHASE 

APS  SAD  IT 


u 

% 


6 

1 

0 

1 

6 

8 

1 

2 

2 

14  •; 

1 

1 

13 

3 

15 

4 

3 

15 

6 

19 

4 

7 

20 

13 

23 

6 

5 

23 

18 

29 

3 

2 

28 

20 

32  i 

2 

2 

32 

22 

34  | 

7 

2 

33 

24 

41  ! 

4 

3 

38 

27 

45 

2 

2 

43 

29 

47 

0 

1 

44 

30 

47 

1 

1 

48 

31 

48 

1 

20 

49 

51 

49 

1 

5 

50 

56 

50 

TOTAL  ERRORS 

DETECTED- 

51 

TOTAL  REMOVED- 

51  j 

DATA  BEGINS 

AT  WEEK  105  OF 

IMPLEMENTATION 

PHASE  1 
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DATA  SETS  FOR  APPLICATION  Of  ^OffwARE^ELli 


iBILITY  MOOELS 


CPCI  CU  PHASE  DETECTED  li^RVAL  CUM.  TIME  ^?ECET^°RS 


APS  SAD  SO 


0 

1 

3 

1 

3 

9 

2 

4 

9 

5 

5 

15 

6 

6 

17 

8 

7 

19 

15 

11 

35 

28 

13 

41 

31 

15 

45 

38 

18 

46 

41 

21 

47 

45 

22 

47 

47 

23 

48 

48 

24 

49 

50 

30 

50 

51 

38 

51 

52 

TOTAL  REMOVED- 

52 

i  OF 

IMPLEMENTATION 

PHASE 

APS  SAD  ST 


0 

1 

0 

1 

2 

3 

3 

3 

3 

4 

6 

10 

10 

10 

12 

13 

15 

15 

16 

18 

19 

17 

20 

19 

18 

22 

19 

20 

TOTAL  REMOVED- 

20 

OF 

IMPLEMENTATION 

PHASE 

V  . 

.  v  A 
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TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERRORS  TIME  CUM  ERRORS  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED  CUM.  TIME  DETECTED 


APS  ZBZ  ST 


3 

1 

0 

1 

3 

1 

1 

3 

2 

4 

2 

1 

4 

3 

6 

2 

1 

5 

4 

8 

5 

2 

7 

6 

13 

11 

3 

12 

9 

24 

2 

1 

23 

10 

26 

5 

1 

26 

11 

31 

0 

4 

27 

15 

31 

4 

3 

32 

18 

35 

6 

4 

35 

22 

41 

0 

5 

41 

27 

41 

TOTAL  ERRORS  DETECTED*  42  TOTAL  REMOVED-  42 
DATA  BEGINS  AT  WEEK  134  OF  IMPLEMENTATION  PHASE 


DRS  DAD  IT 


1 

4 

0 

4 

1 

1 

7 

2 

11 

2 

1 

4 

3 

15 

3 

11 

7 

4 

22 

14 

0 

2 

5 

24 

14 

1 

3 

6 

27 

15 

1 

10 

15 

37 

16 

0 

8 

16 

45 

16 

1 

9 

17 

54 

17 

2 

11 

IS 

65 

19 

0 

6 

19 

71 

19 

TOTAL  ERf.ORS  DETECTED-  20  TOTAL  REMOVED-  20 
DATA  BEGINS  AT  WEEK  95  OF  IMPLEMENTATION  PHASE 


TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


FURORS  TTMF 

CPCI  CU  PHASE  DETECTED  INTERVAL 


OSS  LDR  SD 

10  2 

1  1 

1  1 

10  6 

1  2 

1  7 

0  3 

1  4 

2  S 

0  2 

0  1 


CUM.  ERRORS  CUM.  ERRORS 

REMOVED  CUM.  TIME  DETECTED 


0 

2 

10 

1 

3 

11 

2 

4 

12 

12 

10 

22 

17 

12 

23 

20 

19 

24 

21 

22 

24 

22 

26 

25 

23 

31 

27 

24 

33 

27 

27 

34 

27 

TOTAL  ERRORS  DETECTED-  28  TOTAL  REMOVED-  28 
DATA  BEGINS  AT  WEEK  105  OF  IMPLEMENTATION  PHASE 
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TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


CPCI 


ERRORS 

CU  PHASE  DETECTED 


TIME  CUM.  ERRORS  CUM.  ERRORS 

INTERVAL  REMOVED  CUM.  TIME  DETECTED 


OSS  LDR  ST 


2 

5 

0 

5 

2 

1 

4 

1 

9 

3 

1 

5 

3 

14 

4 

1 

3 

4 

17 

5 

1 

2 

5 

19 

6 

0 

1 

6 

20 

6 

4 

8 

7 

28 

10 

0 

2 

10 

30 

10 

TOTAL  ERRORS 

DETECTED- 

11 

TOTAL  REMOVED- 

11 

DATA  BEGINS 

AT  WEEK  125 

OF 

IMPLEMENTATION 

PHASE 

SES  VAS  IT 


3 

7 

0 

7 

3 

0 

1 

2 

8 

3 

3 

7 

3 

15 

2 

2 

4 

17 

0 

1 

5 

18 

1 

1 

7 

19 

0 

1 

8 

20 

4 

4 

9 

24 

13 

0 

4 

11 

28 

13 

0 

6 

13 

34 

13 

TOTAL  ERRORS  DETECTED-  14  TOTAL  REMOVED-  14 
DATA  BEGINS  AT  WEEK  111  OF  IMPLEMENTATION  PHASE 
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TABLE  4.1  1  (CONTINUED) 

DATA  SETS  FOR  APPLICATION  OF  SOFTWARE  RELIABILITY  HOOELS 


CPC I  CU  PHASE 
SES  VAS  SD 


ERRORS 

TIME 

CUM.  ERRORS 

CUM.  ERRORS 

DETECTED 

INTERVAL 

REMOVED 

CUM.  TIME 

DETECTED 

0 

2 

0 

2 

0 

4 

6 

1 

8 

4 

0 

1 

2 

9 

4 

1 

1 

3 

10 

5 

0 

6 

4 

16 

5 

5 

3 

6 

19 

10 

1 

2 

9 

21 

11 

1 

4 

12 

25 

12 

6 

4 

13 

29 

18 

1 

4 

18 

33 

19 

0 

2 

19 

35 

19 

TOTAL  ERRORS  DETECTED-  20  TOTAL  REMOVED-  20 
DATA  BEQINS  AT  WEEK  105  OF  IMPLEMENTATION  PHASE 


SES  VAS  ST 


2 

3 

0 

3 

1 

1 

1 

4 

1 

2 

3 

6 

0 

8 

4 

14 

TOTAL  ERRORS 

DETECTED- 

5 

TOTAL  REMOVED- 

5 

DATA  BEQINS 

AT  WEEK  138 

OF 

IMPLEMENTATION 

PHASE 

SUS  CON  IT 


1 

1 

0 

3 

3 

0 

1 

5 
1 
1 

6 
6 
1 
0 
0 


1 

3 

4 
63 

5 
8 

12 

7 

3 

1 

2 

3 
1 
1 

4 


0 

1 

2 

3 

5 

6 
7 
9 

12 

13 

IS 

IS 

25 

28 

29 


1 

1 

4 

2 

8 

2 

71 

5 

79 

8 

85 

8 

97 

9 

104 

14 

107 

15 

108 

16 

110 

22 

113 

28 

114 

29 

115 

29 

119 

29 

TOTAL  ERRORS  DETECTED-  30  TOTAL  REMOVED-  30 
DATA  8EQINS  AT  WEEK  3  OF  IMPLEMENTATION  PHASE 
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TABLE  4.1.1  (CONTINUED) 

DATA  SETS  FOB  APPLICATION  OF  SOFTWARE  RELIABILITY  MODELS 


ERRORS  TIME  CUM.  ERRORS 

CPCI  CU  PHASE  DETECTED  INTERVAL  REMOVED 


CUM.  ERRORS 
CUM.  TIME  DETECTED 


SUS  CON  SD 


SUS  CON  ST 


3 

12 

0 

12 

3 

4 

5 

1 

17 

7 

0 

1 

2 

18 

7 

3 

S 

7 

23 

10 

0 

2 

10 

25 

10 

TOTAL  ERRORS 

DETECTED- 

11 

TOTAL  REMOVED- 

11 

DATA  BEGINS 

AT  WEEK  110 

OF 

IMPLEMENTATION 

PHASE 

0 

4 

0 

4 

0 

5 

5 

1 

9 

5 

0 

5 

3 

14 

S 

1 

3 

4 

17 

6 

0 

2 

5 

19 

6 

2 

1 

7 

20 

8 

1 

1 

8 

21 

9 

0 

1 

9 

22 

9 

2 

16 

10 

38 

11 

0 

10 

11 

48 

11 

TOTAL  ERRORS 

DETECTED- 

12 

TOTAL  REMOVED- 

12 

DATA  BEGINS 

AT  WEEK  124 

OF 

IMPLEMENTATION 

PHASE 
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first  error  was  detected.  The  time  at  which  the  first  error  was 
detected  was  therefore  selected  as  the  time  origin  and  the  first 
error  not  counted  in  the  data.  Loss  of  this  information  should 
not  critically  affect  the  fit  of  the  models  overall. 

4.2  Model  Fitting 

Each  software  reliabiliy  model  was  applied  to  each  data¬ 
set  listed  in  Table  4.1.1  of  Section  4.1  and  estimates  of  the 
associated  parameters  attempted.  In  each  attempt,  if  convergence 
of  the  iterative  procedures  was  not  obtained  after  several  at¬ 
tempts  with  different  starting  points,  or  if  convergence  to  in¬ 
valid  parameter  estimates  was  obtained,  the  attempt  was  judged 
unsuccessful.  The  models  were  fit  in  the  following  order: 
Geometric  Poisson,  Jelinski-Moranda,  Nonhomogeneous  Poisson, 
Generalized  Poisson,  IBM  Poisson  (modified).  Binomial,  and  IBM 
Poisson.  For  the  Geometric  Poisson,  a  starting  value  for  K  of 
0.5  was  used,  since  0<K<1.  If  convergence  was  obtained,  then 
Table  3-8.1  was  used  to  translate  the  Geometric  Poisson 
parameters  to  parameter  values  for  the  other  models.  These 
values  were  then  used  as  starting  values  for  the  other  models. 

If  convergence  was  not  obtained  for  a  starting  value  of  K=0.5  for 
the  Geometric  Poisson  model,  then  other  starting  points  from 
(0,1)  were  tried;  e.g.  0.125,  0.25,  0.625,  0.75,  0.875,  etc.  In 
general,  if  convergence  was  not  obtained  for  a  given  model, 
several  sets  of  starting  points  were  tried.  For  example,  if 
iteration  was  performed  on  N,  then  several  starting  values  for  N 
(all  greater  than  the  total  number  of  errors  removed  for  the 
dataset)  were  tried.  Similarly,  for  iteration  on  <f>  (or  b  in  the 
Nonhomogeneous  Poisson  Model)  several  starting  points  ranging 
from  0.001  up  to  1.5  were  tried.  In  general,  if  convergence  was 
obtained  and  valid  parameter  estimates  obtained  for  a  given  mod¬ 
el,  no  effort  was  made  to  try  other  starting  points  in  order  to 
obtain  convergence  to  different  values.  Such  an  investigation, 
although  beyond  the  scope  of  this  study,  should  be  carried  out  in 
the  future  since  the  uniqueness  of  parameter  estimates  has  not 
been  addressed  by  the  respective  authors  of  the  models. 

Whenever  the  iterative  procedures  converged  to  valid 
parameter  values,  it  was  necessary  to  perform  a  statistical  test 
of  fit  for  the  models. 

Very  little  analysis  has  been  performed  by  the  authors 
of  the  models  in  connection  with  testing  their  fit. 

Goel  (1980)  proposed  a  Kolmogorov-Smirnov  test  for  test¬ 
ing  the  fit  of  the  Nonhomogeneous  Poisson  Model  when  the  times 
between  software  errors  are  available.  However,  on  JSS,  the  ex¬ 
act  times  were  not  available,  and  moreover,  the  procedure  advo¬ 
cated  by  Goel  (1980)  is  not  the  correct  use  of  the  Kolmogorov- 
Smirnov  test  since  the  unknown  parameters  are  estimated  from  the 
sample.  Although  this  was  recognized  by  Go^l  (1980),  his  sugges¬ 
ted  approach  of  doubling  the  significance  level  when  choosing  the 


critical  value  has  not  been  shown  to  be  a  valid  approach  for  the 
model  in  question.  A  better  approach  would  have  been  to  Monte 
Carlo  the  sampling  distribution  of  the  Kolmogorov- Smirnov  Statis¬ 
tic  with  the  parameters  estimated  in  order  to  obtain  approximate 
true  critical  values.  This  was  the  approach  in  Schafer  et.al. 
(1979)  in  determining  the  applicability  of  the  classical  Chi- 
square  goodness-of-f it  test  when  software  reliability  model 
parameters  are  estimated  from  the  data.  Since  the  data  on  JSS 
are  available  in  grouped  form  only,  this  procedure  is  appropriate 
for  analyzing  the  fit  of  the  models  to  the  JSS  data. 


Schafer  et.al.  (1979)  observed  that  the  classical  chi- 
square  goodness-of-f it  test  provided  an  adequate  goodness-of-f it 
test  for  the  Jelinski-Moranda,  Generalized  Poisson,  Binomial,  and 
Nonhomogeneous  Poisson  Models  when  their  respective  parameters 
are  estimated  from  the  sample.  In  view  of  the  equivalence  es¬ 
tablished  between  the  Nonhomogeneous  Poisson  Model  and  the 
Geometric  Poisson  Model,  this  goodness-of-f it  test  should  also  be 
valid  for  the  Geometric  Poisson  model.  The  test  procedure  is 
simple;  based  on  errors  detected  in  time  interval  i  of  length 
(1  <.  i  <_  k)  estimate  the  parameters  of  the  software  reliability 
model  in  question  and  form  the  statistic 


X2 


k  (Ni  -  Ei)2 
i=l  Ei 


where  Ei  is  the  expected  value  of  Ni  with  the  parameters  relaced 
by  their  estimates.  Obviously,  large  values  of  X2  indicate 
deviation  from  the  underlying  model.  In  Schafer  et.al.  (1979)  it 
was  shown  that  under  the  assumption  that  the  model  is  actually 
correct,  the  distribution  of  X2  is  approximately  chi-square  with 
k-l-e  degrees  of  freedom  where  e  is  the  number  of  parameters  es¬ 
timated.  Thus,  the  goodness-of-f it  test  is  to  reject  the  valid¬ 
ity  of  the  model  if  the  observed  value  of  X2  exceeds  the  1-Y 
quantile  of  the  chi-square  distribution  with  k-l-e  degrees  of 
freedom.  Here,  Y  is  the  significance  level  of  the  test.  We 
chose  Y  *  0.05  for  every  case. 


Summary  of  Model  Fitting  Attempts 


The  results  of  fitting  the  software  reliability  models 
to  the  data  in  Table  4.1.1  of  Section  4.1  are  summarized  in  Table 
4.3.1.  In  this  table,  the  term  "fit"  means  that  the  model  did 
not  fail  a  chi-square  goodness-of-f it  test  at  the  0.05  level  of 
significance,  while  the  phrase  "lack-of-f it"  signifies  that  the 
model  failed  a  chi-square  goodness-of-f it  test  at  the  0.05  level 
of  significance.  The  phrase  "no  convergence"  means  that  either 
the  iterative  procedures  failed  to  converge  altogether,  or  failed 
to  converge  to  valid  parameter  estimates  (e.g.  negative  N,  or  K>1 
in  the  Geometric  Poisson) . 


Obviously,  since  each  dataset  in  Table  4.1.1  of  Section 
4.1  constitutes  a  unique  CU  and  test  phase,  the  parameter  esti¬ 
mates  are  peculiar  to  the  CU  and  test  phase  on  which  they  are 
based.  For  example,  the  "N"  being  estimated  during  the  IT  phase 
for  a  given  CU  is  not  the  same  "N"  being  estimated  during  the 
next  test  phase  for  the  same  CU  because  errors  were  removed 
during  IT. 

Table  4.3.1 

Summary  of  Model  Fitting  Attempts 


Model  fit  %*  lack  of  fit  %*  no  convergence 


Geometric  Poisson 

33 

25 

42 

Jelinski-Moranda 

18 

8 

74 

Nonhomogeneous 

Poisson 

33 

25 

42 

Generalized 

Poisson 

35 

10 

55 

IBM  Poisson 
(Modified) 

53 

22 

25 

IBM  Poisson 

0 

0 

100 

Binomial 

27 

16 

57 

Average 

28.15 

14.85 

57.00 

*  Percents  based  on  attempted  fits  of  each  model  on 
each  of  51  datasets  contained  in  Table  4.1.1  of  Section 
4.1. 


For  the  purpose  of  comparison.  Table  4.3.2  lists  the 
results  for  each  dataset.  In  this  table,  the  parameter  estimates 
for  each  model  have  been  used  to  calculate  values  for  N  and 
(using  the  conversion  table  3.8.1  in  Section  3.8)  to  allow  easy 
comparisons  between  models.  For  the  Jelinski-Moranda,  General¬ 
ized  Poisson,  IBM  Poisson  and  Binomial  models,  N  is  intrepreted 
as  the  number  of  initial  errors.  Analogously,  for  the  Geometric 
Poisson  and  the  Nonhomogeneous  Poisson  models,  N  is  the  expected 
number  of  errors  to  be  detected  in  infinite  time. 

Each  page  of  Table  4.3.2  represents  the  results  of  ap¬ 
plying  each  model  to  a  particular  dataset  identified  by  CPCI,  CU, 
and  PH  (test  phase).  The  total  number  of  errors  observed  and 
removed  is  printed  below  the  dataset  identifier.  The  table 
entries  are  the  estimate  of  N,  observed  errors  for  purposes  of 
fitting  (this  number  will  be  one  less  than  the^ number  printed 
below  the  dataset  identifier),  the  estimate  of*<£,  the  estimate  of 
(applies  only  to  the  Generalized  Poisson  and  IBM  Poisson  Models) 
the  observed  value  of  the  chi-square  goodness-of-f it  statistic, 
the  degrees  of  freedom  (the  total  number  of  time  intervals  less 
1,  and  less  the  number  of  parameters  estimated),  and  the  95% 
point  of  the  chi-square  distribution  for  testing  goodness-of-f it . 

An  attempt  was  made  to  obtain  fits  for  the  four  consecu¬ 
tive  test  phases  IT  (integration  test),  ST  (system  test),  SD  (in¬ 
dependent  test),  and  IN  (installation  test).  In  some  cases, 
there  was  insufficient  data  to  fit  the  models,  i.e.  not  enough 
time  intervals  could  be  formed  such  that  no  errors-  were  removed 
during  each  time  interval.  These  cases  are  summarized  in  Table 
4.3.3.  Throughout ,  cases  where  there  was  lack  of  convergence  are 
shown  as  entries  containing  asterisks  in  the  fields,  of  simply  by 
omitting  the  model  entry  for  the  dataset. 

Appendix  A  gives  the  detailed  results  of  the  model  ap¬ 
plications  including  the  actual  parameter  estimates  and  a  table 
of  observed  versus  expected  number  of  errors  for  each  time 
period . 
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4.4 


Discussion  of  Results 


The  overall  success  of  applying  the  models  is  summarized 
in  Table  4.3.1  of  Section  4.3-  The  highest  percentage  of  good 
fits  was  achieved  by  the  IBM  Poisson  model  (modified  as  discussed 
in  Section  3.4)  with  53 %  of  the  attempts  resulting  in  fits  (a 
"passed"  goodness-of-f it  test).  Of  the  cases  where  convergence 
was  achieved,  roughly  71%  of  the  attempts  led  to  good  fits  for 
the  modified  IBM  Poisson  Model.  In  direct  contrast,  the  un¬ 
modified,  three  parameter  IBM  Poisson  Model  experienced  no  suc¬ 
cess  with  100%  of  the  attempts  resulting  in  failure  of  the 
parameter  estimating  algorithms  to  converge.  The  Geometric  Pois¬ 
son,  Nonhomogeneous  Poisson,  Generalized  Poisson,  and  Binomial 
Models  experienced  roughly  the  same  success  rates,  while  the 
modified  Jelinski-Moranda  model  had  the  second  to  lowest  success 
rate  at  18%. 

The  comparatively  poor  performance  of  the  Jelinski- 
Moranda  model  is  also  indicative  of  the  failure  of  the  Imperfect 
Debugging  model  in  view  of  the  similarities  between  these  two 
models.  Table  4.4.1  gives  a  rank  ordering  of  the  models  by  suc¬ 
cess  rate.  We  caution  that  "success"  means  that  the  model 
parameter  estimator  algorithms  converged  and  a  good  fit  was  ob¬ 
tained  (i.e.  the  chi-square  test  was  not  failed).  Thus,  since 
failure  of  the  parameter  estimating  algorithms  to  converge  is  not 
necessarily  due  to  failure  of  the  model  to  be  valid  (i.e.  it 
could  be  due  to  lack  of  starting  points  for  the  iterations)  the 
rank  ordering  can  be  misleading.  For  example,  the  IBM  Poisson 
model  is  ranked  highest  with  a  53%  success  rate,  while  the  Jelin¬ 
ski-Moranda  model  is  second  to  lowest  with  18%  success  rate. 
However,  if  only  cases  where  convergence  was  obtained  are  con¬ 
sidered,  the  ranking  would  be  headed  by  the  Generalized  Poisson 
Model  (78%)  followed  by  the  modified  IBM  Poisson  model  (71%),  the 
Jelinski-Moranda  model  (69%),  the  Binomial  model  (64%),  and  the 
Geometric  Poisson  and  Nonhomogeneous  Poisson  models  (both  at 
57%) •  That  the  Generalized  Poisson  model  has  the  highest  per¬ 
centage  of  good  fits  for  cases  in  which  convergence  was  obtained 
was  to  be  expected  since  this  model  has  three  parameters  and  thus 
more  flexibility  to  fit. 

That  the  IBM  Poisson  model  (modified)  showed  the  highest 
overall  success  rate  of  53%  is  also  a  little  misleading.  Refer¬ 
ring  to  Table  4.3.2,  in  nine  of  the  instances  where  a  good  fit 
was  obtained  for  this  model,  ridiculously  large  values  for  N  were 
reported  (e.g.  5,643  for  APS  ASZ  ST)  while  extremely  small  values 
for  <f>  were  reported.  These  cases  reflected  an  identif lability 
problem  in  the  model,  i.e.  the  data  did  not  show  enough  structure 
to  allow  identification  of  both  parameters  N  and  <f> .  If  these 
cases  are  not  counted  as  good  fits,  then  the  overall  success  rate 
would  be  35%  rather  than  53%. 
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Table  4.4.1 


Model  Success  Percentages 


Percentage  of  Good 
Fits  When  Convergence 

Model  Success  Percentage  Was  Achieved 


IBM  Poisson 
(modified) 

53 

71 

Generalized  Poisson 

35 

78 

Nonhomogeneous 

Poisson 

33 

57 

Geometric  Poisson 

33 

57 

Binomial 

27 

64 

Jelinski-Moranda 

18 

69 

IBM  Poisson 

0 

It  is  interesting  to  note  that  the  similarities  noted  in 
Section  3-8  have  manifested  themselves  in  Table  4.3.2  of  Section 
4.3.  That  is,  in  nearly  every  instance  where  the  majority  of  the 
six  models  (Geometric  Poisson,  Jelinski-Moranda,  Nonhomogeneous 
Poisson,  Generalized  Poisson,  IBM  Poisson,  and  Binomial)  showed 
success,  the  estimates  of  N  and  <f>  were  very  close.  Moreover, 
the  equivalence  of  the  Nonhomogeneous  Poisson  and  Geometric  Pois¬ 
son  Models  is  evident;  every  instance  in  which  these  two  models' 
parameter  estimating  procedures  converged,  the  same  value  for  N 
was  obtained  (see  Table  3.8.1)  and  the  correspondence  in  equa¬ 
tions  (3.8.6)  held.  In  view  of  this  correspondence  it  is  not 
surprising  that  the  success  percent  was  the  same  for  both 
models . 


Another  important  result  to  be  inferred  from  Table  4.3.2 
of  Section  4.3  is  related  to  the  test  phase  dependence  of  the 
models.  Earlier  we  discussed  the  fact  that  testing  intensity  and 
methodology  are  roughly  constant  within  a  test  phase,  but  differ 
markedly  between  test  phases.  Moreover,  it  is  conceivable  that 
the  possible  types  of  errors  which  can  be  detected  during  dif¬ 
ferent  test  phases  varies  with  test  phase  as  depicted  in  Figure 
4.4.1.  This  is  a  plausible  explanation  for  the  fact  that  often 
in  Table  4.3*2,  the  number  of  errors  detected  in  one  test  phase 
greatly  exceeds  the  estimated  residual  number  of  errors  predicted 
by  a  model  (one  that  fits)  in  the  previous  test  phase.  For  exam¬ 
ple,  in  the  case  of  CPCI  APS,  CU  AAZ,  the  Generalized  Poisson 
Model  was  a  good  fit  (a  chi-square  observed  as  5*917  on  8  degrees 
of  freedom)  for  the  data  from  the  IT  (integration  test)  phase. 

The  initial  number  of  errors  was  estimated  to  be  25.  Since  20 
errors  were  removed,  the  estimated  number  of  residual  errors 
would  be  5*  During  the  next  test  phase,  SD  (independent  test)  10 
addition  errors  were  removed;  twice  as  many  as  the  estimated  num¬ 
ber  of  residual  errors  from  the  previous  phase.  Once  again,  the 
Generalized  Poisson  Model  was  a  good  fit  on  the  SD  phase  data, 
estimating  N  to  be  16,  leaving  6  as  the  estimated  number  of 
residual  errors.  The  subsequent  number  of  errors  removed  during 
the  ST  (system  test)  and  IN  (installation  test)  phases  was  8+6 
=  14,  more  than  twice  the  estimated  number  of  residual  errors 
estimated  during  SD.  Examples  similar  to  this  abound  in  Table 
4.3*2  of  Section  4.3*  These  examples  do  not,  by  themselves,  in¬ 
dicate  that  the  software  reliability  models  are  inadequate. 
Rather,  they  support  the  conjecture  that  each  test  phase  is 
capable  of  detecting  its  own  (possibly  unique)  class  of  errors. 
They  also,  however,  support  the  unfortunate  conclusion  that  the 
models  are  of  no  use  in  general  for  predicting  errors  from  future 
test  phases.  Table  4.4.2  gives  the  comparisons  of  estimated 
residual  errors  and  errors  actually  removed  for  each  model  and  CU 
for  which  at  least  two  of  the  test  phases  (IT,  SD,  ST,  and  IN) 
resulted  in  good  fits. 

It  is  not  surprising  that  the  overwelming  majority  of 
fits  obtained  were  from  CUs  in  APS  since  APS  generated  the  most 
errors  (nearly  4  times  the  number  from  its  nearest  competitor, 
OSS).  As  mentioned  in  Section  2.6,  the  highest  error  rates  per 
module  were  achieved  by  APS  and  OSS  where  the  percent  of  "lifted" 
design  was  more  balanced  with  the  percent  of  newly  developed 
code.  We  are  not  able  to  draw  any  conclusions  concerning  the 
relationship  between  newly  developed  design  percent  and  the  ten¬ 
dency  of  the  models  to  fit  because  the  data  from  the  other  CPCIs 
is  so  sparse. 


MODEL  PREDICTIONS  FOR  CPCI  APS 


Residual  Errors  Residual  Errors  Residual  Errors  Residual  Errors 


5-  CONCLUSIONS  AND  RECOMMENDATIONS 

5 . 1  Software  Error  Data  Collection 


The  software  error  data  collection  guidelines  proposed 
in  Section  2.*l  are  sound  and  should  be  applicable  to  any  c’l 
software  project  on  which  high  quality  data  on  software  errors  is 
needed.  An  error  data  collection  system  like  the  one  employed  in 
collecting  the  JSS  data  is  relatively  inexpensive  to  implement 
(about  0.6%  of  the  overall  software  development  effort  on  JSS) 
and  use,  and  provides  the  requisite  input  data  for  the  software 
reliability  models  (except  for  the  Imperfect  Debugging  Model  in 
its  original  form)  described  in  this  report.  Whether  or  not  the 
data  is  used  for  fitting  these  software  reliabilty  models,  such  a 
database  can  be  an  invaluable  source  of  historical  data  to  aid  in 
both  the  project  and  acquisition  manager's  decision  processes. 

It  is  however,  important  to  note  some  characteristics  of 
a  database  compiled  like  the  JSS  database  in  relation  to  the  as¬ 
sumptions  made  in  many  software  reliability  models.  First,  the 
data  is  "coarse"  in  the  sense  that  the  time  scale  is  calendar 
time  and  the  dates  when  errors  are  detected  (and,  resolved  and 
verified)  can  be  determined  only  to  the  week.  However,  on  a 
project  the  size  of  JSS,  keeping  track  of  execution  time,  for 
example,  would  have  severely  perturbed  the  project  in  general. 

Secondly,  software  testing  does  not  cease  when  an  error 
is  detected  and  resume  when  the  error  is  removed.  Such  a  re¬ 
quirement  would  severely  delay  the  development  of  a  large  scale 
project  like  JSS.  This  assumption  is  commonly  made  in  the 
original  forms  of  many  software  reliability  models. 

Thirdly,  the  time  to  remove  an  error  is  almost  never 
"negligible",  (another  assumption  commonly  made  in  software 
reliability  modelling). 

Next,  all  the  software  is  not  under  test  at  all  times. 

In  fact,  not  all  the  software  in  a  given  module  is  under  test 
when  the  module  is  under  test .  This  is  in  direct  violation  to 
the  assumptions  (either  implicit  or  explicit)  of  all  the  software 
reliability  models  studied  here. 

Finally,  calendar  time  is  not  necessarily  representative 
of  debugging  time  in  view  of  varying  manpower,  testing  intensity 
(test  phase),  and  schedule  milestones.  Thus,  any  time-dependent 
figures-of -merit  derived  from  any  of  the  software  reliability 
models  must  be  suspect  if  used  to  represent  the  operational  sce¬ 
nario  of  the  software. 


It  is  the  opinion  of  many  software  managers  at  Hughes 
that  to  "upgrade"  the  software  error  data  collection  efforts  on 


large  projects  to  meet  the  assumptions  of  the  software  reliabili¬ 
ty  models  (in  their  original,  unmodified  forms)  would  incur  un¬ 
reasonable  costs;  costs  which  could  better  be  spent  during  soft¬ 
ware  design.  We  believe  that  a  database  (like  JSS)  developed 
within  the  guidelines  proposed  in  Section  2.4,  provides  the  best 
software  error  data  to  be  expected  within  a  reasonable  budget, 
does  not  result  in  project  delays,  and  is  relatively  easy  to  man¬ 
age.  That  it  would  not  be  ideally  suited  for  software  reliabili¬ 
ty  modelling  is  neither  surprising  nor  disturbing,  for  the 
majority  of  the  data  collection  system  on  which  it  is  based  was 
developed  long  before  software  reliability  models  emerged,  and 
therefore  not  with  software  reliability  prediction  as  its  pur¬ 
pose.  Nevertheless,  the  data  collected  under  this  system  will 
accommodate  most  of  the  current  models,  and  before  major  changes 
to  this  data  collection  system  are  undertaken,  a  definitive  soft¬ 
ware  reliability  modeling  methodology  must  be  developed.  In  view 
of  these  considerations,  we  recommend  that  software  error  data  on 
large  CPI  projects  be  collected  as  described  in  Section  2.4  and 
that  future  attempts  at  validating  a  definitive  software 
reliability  model  be  performed  on  small-scale,  ad  hoc  computer 
programs  (as  done  by  Nagel  and  Skrivan  (1982)).  In  addition,  we 
recommend  that  for  large  C3I  projects,  that  manpower  loading  in 
the  software  debugging  effort  be  continuously  monitored,  and  that 
the  exact  date  at  which  each  unit  of  software  begins  each  new 
test  phase  be  recorded.  When  a  definitive  software  reliability 
model  is  discovered  under  these  circumstances,  then  the  issue  of 
more  comprehensive  data  collection  on  the  larger-scale  (PI  proj¬ 
ects  can  be  effectively  and  economically  addressed. 

5 • 2  Software  Reliability  Models  and  Guidelines  for  their 

Use  by  Software  Acquisition  Managers. 

Early  in  the  course  of  this  study  we  discovered  that 
several  of  the  important  assumptions  made  by  the  software 
reliability  models  considered  in  this  study  are  not  valid  for  the 
JSS  project,  and  probably  not  true  in  general.  Also,  disappoint¬ 
ingly,  the  prospect  of  collecting  the  times  between  error  detec¬ 
tions  was  dismissed  early  as  impractical,  making  the  use  of  the 
original  Imperfect  Debugging  Model  impossible  (the  Jelinski- 
Moranda  model  was  used  instead  because  of  its  similarity  to  the 
Imperfect  Debugging  Model).  These  assumptions  which  are  violated 
have  been  pointed  out  in  the  text  of  this  report.  Some  of  them 
are  of  a  mathematical  nature  (e.g.  each  error  occurs  with  the 
same  rate)  and  some  are  related  to  the  data  collection  (e.g.  that 
each  error  is  immediately  removed  when  detected  or  that  removal 
time  is  negligible,  testing  stops  while  the  error  is  removed, 
testing  is  uniform,  all  the  software  is  being  tested,  etc).  By 
restricting  model  fitting  attempts  to  single  compilation  units 
(CUs)  and  test  phases,  and  modifying  some  of  the  models,  we  al¬ 
leviated  most  of  the  violations  relating  to  data  collection  and 
thus  provided  these  models  a  better  chance  to  fit  the  JSS  data. 
Indeed,  there  would  have  been  no  point  in  fitting  the  models  in 
their  original  form  to  all  the  JSS  data  at  once  since  they  were 


clearly  not  valid  for  the  JSS  data  overall.  The  models  which 
were  modified  mathematically  were  the  IBM  Poisson  Model,  and  the 
Geometric  Poisson  Model.  Only  for  the  IBM  Poisson  Model  could 
both  the  original  and  modified  versions  be  utilized,  and  the 
result  was  that  the  modified  version  showed  the  highest  percent 
of  good  fits  over  all  models,  while  the  original  version  failed 
in  every  attempt  to  achieve  parameter  estimates . 

In  spite  of  modifications  and  careful  use  of  the  JSS 
data,  the  models  performed  very  poorly  overall  with  respect  to 
application  to  the  JSS  data.  In  particular,  lack  of  convergence 
of  the  iterative  procedures  for  estimating  the  parameters  or  the 
failure  of  chi-square  goodness-of-f it  tests  at  the  0.05  sig¬ 
nificance  level  was  the  rule,  rather  than  the  exception. 

The  similarities  which  exist  among  the  models  are 
surprising.  Roughly  speaking,  they  are  all  slight  variations  on 
the  same  theme;  that  theme  being  constant  and  equal  single  error 
occurrence  rate,  and  that  the  expected  number  of  errors  to  be 
detected  in  a  time  interval  is  proportional  to  the  number  of  er¬ 
rors  remaining  or  "at  risk".  In  fact,  the  Geometric  Poisson  Mod¬ 
el  (as  derived  in  this  report  to  handle  unequal  time  intervals) 
was  shown  to  be  equivalent  to  the  Nonhomogeneous  Poisson  Model 
when  the  time  intervals  are  of  integer  length.  A  quantitative 
assessment  of  how  similar  the  models  are  is  offered  in  Table 
4.3.2  of  Section  4.3.  That  is,  when  all  the  models  fit  a  par¬ 
ticular  dataset,  the  estimates  they  provided  for  the  initial  num¬ 
ber  of  errors  (or  expected  number  of  errors  detectable  in  in¬ 
finite  time,  as  appropriate)  were  not  substantially  different, 
and  often  equal. 

We  found  that  residual  errors  (or  expected  residual  er¬ 
rors,  as  the  case  may  be)  would  be  the  most  appropriate 
reliability  measure  to  project  personnel  because  of  its  time 
scale  independence.  Other  time-dependent  figures-of -merit  can  be 
misleading  when  based  on  a  model  fit  to  calendar  time  data  since 
calendar  time  is  not  uniformly  representative  of  test  phase  time 
nor  system  operation  time. 

As  far  as  the  predictive  capability  of  the  models,  the 
evidence  suggests  that  the  models  cannot  predict  the  number  of 
residual  errors  detectable  in  future  test  phases.  The  failure  of 
the  models  to  accurately  predict  the  number  of  errors  remaining 
was  so  dramatic,  in  fact,  that  detailed  statistical  analyses  of 
these  predictions  were  not  necessary.  The  explanation  for  this 
is  not  necessarily  the  failure  of  the  models  and  their  assump¬ 
tions.  Rather,  we  believe  that  this  failure  is  partly  due  to  the 
fact  that  different  test  phases  for  software  can  detect  their  own 
different  (often  dramatically  different)  classes  of  errors. 

While  these  classes  obviously  overlap,  they  can  be  very  different 
indeed.  For  example,  some  errors  in  module  interfaces  may  not  be 
detectable  prior  to  parameter  and  assembly  testing,  some  errors 
in  software  subsystem  functions  cannot  all  be  detected  prior  to 
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independent  testing,  and  so  on.  Thus,  since  the  models  are  ig¬ 
norant  of  this  possibility,  results  obtained  from  data  prior  to  a 
given  test  phase  can  have  very  little  predictive  capability  per¬ 
taining  to  that  test  phase.  On  the  other  hand,  the  results  of 
fitting  the  models  within  a  single  test  phase  appeared  to  be  con¬ 
sistent  with  the  errors  detected  and  removed  within  that  test 
phase,  whenever  the  models  passed  the  goodness-of-fit  test. 

Perhaps  the  most  damaging  aspects  of  the  models  from  the 
point  of  view  of  the  acquisition  manager  are  the  numerical  dif¬ 
ficulties  encountered  in  applying  the  models.  The  issues  con¬ 
cerning  starting  points  for  the  iterative  procedures,  uniqueness 
of  the  estimates,  and  even  alternative  estimation  techniques  must 
be  studied  and  such  problems  solved  before  these  models  can  be 
used  by  acquisition  managers. 

From  the  point  of  view  of  the  Software  Acquisition  Man¬ 
ager,  the  overwelming  difficulty  in  applying  and  obtaining  good 
fits  with  these  models  on  the  JSS  data  should  discourage  their 
use  by  contractual  requirement.  However,  if  in  spite  of  the 
evidence  presented  in  this  study  pertaining  to  the  lack  of  ap¬ 
plicability  of  the  models  to  the  JSS  data,  it  is  determined  that 
one  of  these  models  must  be  used,  we  recommend  the  following 
guidelines  for  their  use: 

a)  Collect  data  according  to  the  guidelines 
in  Section  2.4. 

b)  Apply  the  model  at  the  compilation  unit  level. 

c)  Apply  the  model  to  data  within  a  single  test 
phase,  and  interpret  the  results  in  the  con¬ 
text  of  that  test  phase  only. 

d)  Use  the  results  to  decide  if  more  testing 
within  that  phase  is  necessary. 

e)  Do  not  use  the  results  of  a  model  if,  in 
fitting  the  model,  it  fails  the  chi-square 
goodness-of-fit  test  at  an  appropriate  level 
of  significance  (we  recommend  0.05). 

Guideline  d)  deserves  some  comment.  We  believe  that  the 
best  use  of  one  of  these  models  is  to  compute  the  estimated 
residual  errors  using  Table  3.11.1  of  Section  3.11.  If  this  num¬ 
ber  is  too  large,  then  further  testing  in  that  test  phase  can  be 
recommended.  If  the  residual  error  estimate  is  small,  then  the 
CU  may  proceed  to  the  next  test  phase  (of  course,  the  CU  should 
not  proceed  to  the  next  test  phase  until  all  the  observed  errors 
have  been  removed ) . 

In  conclusion,  we  feel  that  there  is  substantial 
evidence,  both  from  this  study,  and  from  the  study  of  Nagel  and 


Skrivan  (1982)  to  discount  the  general  applicability  to  C^i  proj¬ 
ects  of  the  software  reliability  models  studied  herein,  and  we 
strongly  urge  that  none  of  them  be  adopted  in  any  way  as  industry 
or  government  standards  for  C^i  projects  like  JSS.  Moreover,  we 
view  these  models  as  inappropriate  for  use  by  software  acquisi¬ 
tion  managers  in  monitoring  project  status  and  the  results  of 
qualification  testing. 

5 • 3  Recommendat ions 

While  we  cannot  recommend  the  software  reliability  mod¬ 
els  studied  in  this  report  for  general  use,  there  are  some  alter¬ 
native  modeling  techniques  which  are  potentially  useful  for  soft¬ 
ware  error  data.  These  techniques  are  well-established  and 
heavily  used  in  all  aspects  of  engineering.  A  compelling  ad¬ 
vantage  to  these  techniques  is  that  mathematical  software  has 
been  developed  for  them  in  most  major  scientific  subroutine 
packages.  These  techniques  are  those  of  regression  analysis,  and 
time-series  analysis. 

Of  course,  before  these  techniques  can  be  used,  it  must 
be  decided  what  quantities  are  of  interest.  Clearly,  no  rela¬ 
tively  simple  mathematical  model  can  encompass  all  the  informa¬ 
tion  concerning  the  effectiveness  of  software  qualification  test¬ 
ing.  The  acquisition  manager  must  become  actively  involved  in 
evaluating  the  testing  techniques  used  to  ensure  that  they  are  up 
to  date  and  qualitatively  adequate  for  the  purposes  of  the  soft¬ 
ware  project  under  consideration.  Having  done  this,  the  acquisi¬ 
tion  manager  must  decide  which  raeasureable  attributes  of 
qualification  testing  are  important  to  monitor  and  predict.  This 
is  the  point  at  which  regression  and/or  time-series  analysis  can 
be  possibly  successfully  applied. 

For  example,  suppose  that  the  cumulative  number  of  er¬ 
rors  detected  during  testing,  and  the  cumulative  number  of  errors 
removed  are  to  be  monitored.  It  is  not  difficult  to  select  a 
regression  function  which  will  follow  the  shape  of  a  typical  plot 
of  cumulative  errors  removed  or  detected  versus  time  (calendar 
time)  as  seen  in  Figures  3.9-1  through  3-9.12  of  Section  3.9. 

The  fitting  of  such  regression  functions  can  provide  short-run 
predictions  of  errors  detected  or  removed,  or  of  the  point  at 
which  errors  removed  will  equal  errors  detected,  for  example. 
After  sufficient  time  has  elapsed,  an  estimate  for  the  total  ini¬ 
tial  errors  can  be  obtained  if  the  regression  function  has  a 
mathematical  asymptote  representing  this  value. 

A  more  appropriate  technique  if  predictions  are  required 
would  be  that  of  the  time-series  approach.  While  this  technique 
cannot  be  fully  described  here,  suffice  it  to  say  that  the  ap¬ 
proach  aims  at  identifying  the  underlying  stochastic  structure 
which  relates  successive  observations,  estimating  any  unknown 
parameters,  and  using  the  stochastic  structure  to  make  predic¬ 
tions  for  the  future.  The  amount  of  time  into  the  future  for 
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which  meaningful  predictions  can  be  made  depends  on  the  complexi¬ 
ty  of  the  underlying  stochastic  structure  (see  Box  &  Jenkins, 
1970). 


If  stochastic  models  are  needed  for  monitoring  the  num¬ 
ber  of  software  errors  detected  or  removed,  then  these  approaches 
(i.e.  regression  and  time-series)  are  certainly  worth  looking 
into.  Other  quantitative  methods  may  also  provide  aid  to  the 
acquisition  manager  such  as  the  use  of  software  quality  metrics. 
More  research  is  needed,  however,  to  determine  what  relationship, 
if  any,  exists  between  software  quality  metrics  and  software 
reliability. 

There  is  another  important  recommendation  concerning 
quantitative  software  reliability  modeling.  In  past  attempts  the 
same  models  proposed  have  been  assumed  to  model  software  failures 
during  all  phases  of  the  software’s  development  and  operation. 
There  is  convincing  evidence,  both  from  the  results  of  this  study 
and  from  common  sense,  that  the  detection  of  errors  is  test  phase 
dependent.  In  fact,  during  testing,  the  detection  of  software 
errors  may  well  be  just  as  much  a  function  of  factors  extraneous 
to  the  software  (e.g.  manpower,  scheduling,  test  phase,  testing 
intensity,  programmer  experience)  as  of  the  software  itself. 

Thus,  we  recommend  a  dichotomy  in  future  modeling  attempts;  one 
model  or  technique  for  monitoring  quantities  of  interest  during 
testing,  and  a  different  model  or  technique  for  application 
during  mission  operations.  We  further  recommend  that  any  future 
modeling  studies  be  based  on  a  thorough  analysis  of  the  causes 
and  characteristics  of  software  errors  (during  both  testing  and 
mission  operations)  rather  than  an  unmotivated  (except  by  mathe¬ 
matical  convenience)  set  of  assumptions.  At  the  very  least,  any 
new  models  should  be  consistent  with  the  empirical  evidence  col¬ 
lected  thus  far.  For  example  Goel  (1983),  recognizing  that  em¬ 
pirically  the  cumulative  error  detected  curve  changes  inflection 
and  is  not  concave  downward  everywhere  proposed  a  Nonhomogeneous 
Poisson  model  with  a  new  mean  value  function  of  the  form  a(l- 
exp(-btc)>,  a>0,b>0,c>0.  This  is  a  step  in  the  right  direction. 
Also,  the  proportional  hazards  model  studied  in  Nagel  and  Skri- 
van  (1982)  is  a  good  candidate  for  further  study  as  a  model  ap¬ 
propriate  for  the  development  phase  since  it  allows  the  inclusion 
of  "covariates"  which  could  include  manpower,  test  phase,  inten¬ 
sity  of  testing,  etc. 

Concerning  the  models  investigated  in  this  study,  there 
is  much  more  work  to  be  done  in  the  area  of  parameter  estimation 
if  these  models  are  to  receive  any  further  serious  consideration. 
Such  problems  as  estimation  techniques,  starting  points  for 
iterative  procedures,  and  uniqueness  of  estimators  must  be 
addressed . 

Several  follow-up  studies  can  be  recommended.  First, 
the  experiment  done  by  Nagel  &  Skrivan  (1982)  should  be  repeated. 
At  the  very  least,  their  data  should  be  analyzed  using  different 


techniques  in  order  to  strengthen  their  conclusions,  or  provide 
incentive  to  perform  another  similar  experiment.  Secondly,  the 
JSS  software  error  data  collection  should  continue  in  order  to 
obtain  more  operational  data.  This  will  facilitate  future  analy¬ 
sis  of  the  data  for  both  software  reliability  models  and  software 
metrics.  Finally,  other  software  projects  (on  the  scale  of  JSS) 
should  be  studied  to  determine  if  the  conclusions  of  this  study 
will  hold  true,  and  if  they  are  peculiar  to  the  JSS  project  only. 

The  final  recommendation  concerns  software  standardiza¬ 
tion.  Much  of  the  success  in  hardware  reliability  modeling  is 
due  to  the  fact  that  so  many  electronic  parts  are  standard.  It 
is  believed  that  in  modern  C^I  software,  many  functions  (modules, 
perhaps)  can  also  be  standardized.  Of  course,  there  has  not  been 
(until  the  introduction  of  Ada)  an  acceptable  industry  standard 
programming  language,  which  would  be  a  natural  prerequisite  to 
standardization.  It  is  believed  by  many  software  experts  that 
standardized  software  could  lead  to  a  quantum  improvement  in 
software  quality/reliability .  Hughes -Fullerton  is  currently  in¬ 
volved  in  software  standardization  studies  (e.g.  see  Cooper, 

1981;  Andrulaitis,  1981),  and  this  area  certainly  deserves  fur¬ 
ther  research. 


APPENDIX  A 


Detailed  Results  of  Modeling  Fitting  Attempts 


This  appendix  contains  the  detailed  results  of  the  soft¬ 
ware  reliability  model  fitting  attempts.  Each  dataset  is 
specified  by  CPCI,  CU,  and  test  phase  (e.g.  APS  APC  IT).  For 
each  dataset,  each  of  the  seven  models  are  applied:  Geometric 
Poisson  (modified),  Jelinski-Moranda,  Nonhomogeneous  Poisson, 
Generalized  Poisson,  IBM  Poisson  (modified).  Binomial,  and  the 
IBM  Poisson  with  Variable  Alpha  (original  IBM  Poisson).  The  out¬ 
put  for  a  model  consists  of  the  model  name,  its  parameter  esti¬ 
mates  and  the  number  of  iterations  required  in  the  iterative  al¬ 
gorithms,  the  time  interval  lengths,  observed  number  of  errors, 
expected  number  of  errors  (estimated),  the  standard  deviation 
(estimated)  of  the  number  of  errors,  the  observed  chi-square 
value,  and  the  0.95  quantile  of  the  chi-square  distribution  with 
appropriate  degrees  of  freedom.  Sometimes  an  output  is  bypassed. 
These  are  cases  where  convergence  failed  and  parameter  estimates 
were  not  obtained.  Also,  sometimes  a  dataset  is  bypassed.  These 
are  datasets  with  insufficient  data,  i.e.  datasets  for  which 
there  were  not  enough  time  intervals  in  which  no  errors  were 
removed . 


APS  A PC  IT 


GEOMETRIC  POISSON 

THE  ESTIMATE  OF  K=0. 9944210+00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBDA=0. 421118D+00 
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SO(CONO) 

1.70 
2.84 
1.66 
0.63 
1. 01 
1.15 
1.39 
0.56 
0.79 
0.56 
0.96 
0.78 
0.77 
0.94 
1.32 
2.48 
0.99 
1.46 


DEGREES  OF  FREEDOM:  26.3011 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  75 
THE  ESTIMATE  OF  B=  0.55950-02 
NUMBER  OF  ITERATIONS:  1 
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CHI-SQUARE5 

84.9987 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  16  DEGREES  OF  FREEOOM5  26.3011 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 2893290-04 
NUMBER  OF  ITERATIONS5  6 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS5  4355 
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CHI-SQUARE=0.858949D+02 

0.950  qUANTILE  FOR  CHI-SQUARE  WITH  16  DEGREES  OF  FREEDOM5  26.3011 


BINOMIAL 


THE  ESTIMATE  OF  N=  11 
THE  ESTIMATE  OF  A=  0.31220-11 
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0.950  qUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM: 
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GENERALIZED  POISSON 

IBM  POISSON 

THE  ESTIMATE  OF  PHI=-. 1380970*00 
NUMBER  OF  ITERATIONS:  n 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS=  3 


TAU 

OBS 

EXPECTED ( COND) 

SO(COND) 
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CHI-SQUARE=0. 1745880*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM: 


BINOMIAL 


THE  ESTIMATE  OF  N:  17 

THE  ESTIMATE  OF  A=  0.14170*00 
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0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM: 


IBM  POISSON  WITH  VARIABLE  ALPHA 


7.8167 


7.B167 


7.81*7 


APS  APC  IN 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9657330+00 
NUMBER  OF  ITERATIONS5  5 
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CHI-SQUARE50 . 7742800+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  49 
THE  ESTIMATE  OF  B=  0.34870-01 
NUMBER  OF  ITERATIONS5  2 


TAU  OBS  EXPECTED! COND )  SD(CONO) 


1.00  2 

1.00  1 

6.00  9 

2.00  1 

2.00  4 

1.00  3 

2.00  0 


1.68 

1.63 

8.65 

2.50 

2.33 

1.11 

2.10 


1.30 

1.27 

2.94 

1.58 

1.53 

1.05 

1.45 


CHI-SQUARE=  7.7428 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM5 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  ALPHA5  0.7964D+00 
THE  ESTIMATE  OF  PHI5  0.1338D+00 
NUMBER  OF  ITERATIONS5  13 


TAU  OBS  EXPECTEO(COND)  SO (COND) 


2 


THE  CUMULATIVE  NUMBER  OF  ERRORS  OB3ERVED50. 200000D+02 

CHI-SQUARE5  7. 

9205 

P 

0.950  QUANTILE  FOR 

CHI-SQUARE  WITH 

4  DE6REES  OF  FREEDOM5 

9.4917 

ft 

IBM 

POISSON 

i 

THE  ESTIMATE  OF  PHI=0. 3991170-04 
NUMBER  OF  ITERATIONS5  10 

i 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS5  16399 

TAU  OBS 

EXPECTED (CONO) 

SD(CONO) 

*S-’  * 

1.00  2 

1.33 

1.16 

ftai 

1.00  1 

1.33 

1.16 

HR 

6.00  9 

8.00 

2.83 

PT3 

2.00  1 

2.67 

1.63 

2.00  4 

2.67 

1.63 

1.00  3 

1.33 

1.15 

ft 

2.00  0 

2.66 

1.63 

CHI-SQUARE=0 . 6998610+01 

i 

0.950  QUANTILE  FOR 

CHI-SQUARE  WITH 

5  DEGREES  OF  FREEDOM5 

11.0733 

BINOMIAL 

THE  ESTIMATE  OF  N= 

5 

THE  ESTIMATE  OF  A- 

-0.2252D-13 

P 

Nv.'. 

TAU  OBS 

EXPECTED! CONO) 

SD(COND) 

»  • 

f\v 

1.00  2 

-0.00 

0.00 

1.00  1 

-0.00 

0.00 

•ft 

6.00  9 

-0.00 

0.00 

L"  «• 

2.00  1 

0.00 

0.00 

tL»\ 

2.00  4 

0.00 

0.00 

0 

1.00  3 

0.00 

0.00 

\V, 

2.00  0 

0.00 

0.00 

»  '  + 

p  ‘ . 

[’•  •' 

.  *  , 

0.950  QUANTILE  FOR 

fffttftf 

CHI-SQUARE  WITH 

5  DEGREES  OF  FREEDOM5 

11.0733 

tv*. 

p-c 

IBM 

POISSON  WITH  VARIABLE  ALPHA 

.  VVA 


••  \  "V  \ 


APS  ZEZ  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 1027470+01 
NUMBER  OF  ITERATIONS:  6 

THE  ESTIMATE  OF  LAMBDA=0. 276 9140-01 


TAU 

OBS 

EXPECTED (COND) 

56.00 

0 

3.59 

18.00 

1 

2.89 

13.00 

12 

3.16 

2.00 

3 

0.59 

1.00 

3 

0.31 

3.00 

1 

0.98 

2.00 

1 

0.70 

4.00 

1 

1.51 

18.00 

1 

9.27 

CHI-SQUAREsO. 7043540+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SD(CONO) 

1.89 

1.70 

1.78 

0.77 

0.56 

0.99 

0.84 

1.23 

3.04 


7  OEGREES  OF  FREEDOM:  14.0702 


JELINSKI-MORANOA 


ly'V 

w 


kv* 


pp 

••'.v 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  0 

THE  ESTIMATE  OF  B:-0. 27100-01 
NUMBER  OF  ITERATIONS:  3 


TAU 

OBS 

EXPECTED! COND) 

SO! COND 

56.00 

0 

3.59 

1.89 

18.00 

1 

2.89 

1.70 

13.00 

12 

3.16 

1.78 

2.00 

3 

0.59 

0.77 

1.00 

3 

0.31 

0.56 

3.00 

1 

0.98 

0.99 

2.00 

1 

0.70 

0.84 

4.00 

1 

1.51 

1.23 

18.00 

1 

9.27 

3.04 

CHI-SQUARE=  70.4354 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  OEGREES  OF  FREEDOM:  14.0702 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N:  28 


3/4 


HD-R138  196  RELIABILITY  MODEL  DEMONSTRATION  STUDY  VOLUME  1(U) 

HUGHES  AIRCRAFT  CO  FULLERTON  CR  GROUND  SYSTEMS  GROUP 
J  E  ANGUS  ET  AL.  AUG  83  RADC-TR-83-207-VOL-1 
UNCLASSIFIED  F30602-80-C-0273 


F/G  9/2 


NL 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  OF  STANDARDS- 1963-A 


THE  ESTIMATE  OF  ALPHA=-0. 14430*00 
THE  ESTIMATE  OF  PHI=  0.20720*00 
NUMBER  OF  ITERATIONS:  7 


TAU 

OBS 

EXPECTEOI CONDI 

SOI  CONDI 

56.00 

0 

3.21 

1.79 

18.00 

1 

3.64 

1.91 

13.00 

12 

3.68 

1.92 

2.00 

3 

4.63 

2.15 

1.00 

3 

3.46 

1.86 

3.00 

1 

1.53 

1.24 

2.00 

1 

1.25 

1.12 

4.00 

1 

0.96 

0.98 

18.00 

1 

0.64 

0.80 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVEDsO. 2300000*02 


CHI-SQUARE=  25.0589 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DE6REES  OF  FREEDOM:  12.5961 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 8158750-02 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS=  37 


TAU 

OBS 

EXPECTEOI CONDI 

so  t  core 

56.00 

0 

13.51 

3.68 

18.00 

1 

4.90 

2.21 

13.00 

12 

3.51 

1.87 

2.00 

3 

0.55 

0.74 

1.00 

3 

0.21 

0.46 

3.00 

1 

0.43 

0.66 

2.00 

1 

0.26 

0.51 

4.00 

1 

0.47 

0.69 

18.00 

1 

1.88 

1.37 

CHI-SQUARE=0. 8918290*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM:  14.0702 


BINOMIAL 


THE  ESTIMATE  OF  N=  4 

THE  ESTIMATE  OF  A=-0. 21180-12 


TAU  OBS  EXPECTED! CONDI  SOI  CONDI 


CHI-SQUAB 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM* 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  ZEZ  SO 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9795950*00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMBDA=0.315432D+01 


TAU 

OBS 

EXPECTED (CONO) 

SO( CONO 

3.00 

0 

9.27 

3.04 

6.00 

19 

16.91 

4.11 

1.00 

1 

2.62 

1.62 

1.00 

2 

2.57 

1.60 

2.00 

5 

4.98 

2.23 

1.00 

4 

2.41 

1.55 

3.00 

20 

6.95 

2.64 

1.00 

S 

2.22 

1.49 

1.00 

1 

2.18 

1.48 

3.00 

9 

6.27 

2.50 

2.00 

0 

3.97 

1.99 

2.00 

3 

3.81 

1.95 

2.00 

1 

3.65 

1.91 

3.00 

0 

5.21 

2.28 

CHI-SQUARE*0. 6435550+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  12  DEGREES  OF  FREEDOM* 


JEUNSK1-M0RANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  155 
THE  ESTIMATE  OF  B*  0.20620-01 
NUMBER  OF  ITERATIONS*  2 


TAU  OSS  EXPECTED (CONO)  SO(CONO) 


TAU 

oes 

EXPECTED! COND) 

SO! COND 

1.00 

0 

2.03 

1.43 

3.00 

3 

5.29 

2.30 

2.00 

5 

2.94 

1.71 

3.00 

8 

3.68 

1.92 

1.00 

1 

1.06 

1.03 

6.00 

5 

6.21 

2.49 

4.00 

2 

1.99 

1.41 

1.00 

0 

0.41 

0.64 

4.00 

1 

1.39 

1.16 

CHI-SQUARE* 

10. 

2941 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  0E6REES  OF  FREEDOM* 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  27 
THE  ESTIMATE  OF  ALPHA*  0.11800+01 
THE  ESTIMATE  OF  FHI*  0.00650-01 
NUMBER  OF  ITERATIONS*  7 


TAU 

OBS 

EXPECTED! COND) 

SO! COND 

1.00 

0 

1.64 

1.26 

3.00 

3 

5.77 

2.40 

2.00 

5 

3.44 

1.65 

3.00 

8 

4.22 

2.05 

1.00 

1 

0.79 

0.69 

8.00 

5 

6.37 

2.52 

4.00 

2 

1.57 

1.25 

1.00 

0 

0.25 

0.50 

4.00 

1 

0.95 

0.97 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVEO* 0.2500 000+ 02 

CHI-SQUARE*  7.7606 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  OE6REES  OF  FREEDOM* 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*0. 7237630-01 
NUMBER  OF  ITERATIONS*  6 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BU6S* 


29 


CHI-SQUARE=0. 7718460*01 

0.450  QUANTILE  FOR  CHI-SQUARE  WITH 


7  DEGREES  OF  FREEDOMS 


14.0702 


BINOMIAL 


THE  ESTIMATE  OF  N=  30 
THE  ESTIMATE  OF  As  0.6891D-01 


TAU 

OBS 

EXPECTED (COND) 

SO (COND 

1.00 

0 

1.99 

1.41 

3.00 

3 

5.59 

2.36 

2.00 

5 

3.47 

1.86 

3.00 

8 

4.10 

2.02 

1.00 

1 

0.93 

0.96 

8.00 

5 

5.48 

2.34 

4.00 

2 

1.91 

1.38 

1.00 

0 

0.40 

0.63 

4.00 

1 

1.43 

1.20 

CHI-SQUARE- 

8. 

1655 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DE6REES  OF  FREEDOMS 


14.0702 


APS  ZEZ  IN 


IBM  POISSON  WITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 8344040*00 
NUMBER  OF  ITERATIONS2  4 

THE  ESTIMATE  OF  LAH80A=0. 1595960*01 


TAU 

OBS 

EXPECTED (COND) 

SOI COND 

3.00 

5 

4.04 

2.01 

2.00 

0 

1.70 

1,30 

2.00 

1 

1.18 

1.09 

5.00 

3 

1.62 

1.27 

3.00 

0 

0.46 

0.68 

CHZ-SQUAREsO. 3602810*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


JELINSKI-MORANOA 


3  DEGREES  OF  FREEDOMS 


7.8167 


.  fc'»  .  _  .  _  ( 

.'.'-V.-.A.V.'.’-V'.-.V.'A'.V'AA'jki 


V  V. 


NONHOMOGENE OUS  POISSON 


THE  ESTIMATE  OF  N=  10 
THE  ESTIMATE  OF  B=  0.16100*00 
NUMBER  OF  ITERATIONS*  3 


EXPECTED (COHO) 


SO!  CONDI 


CHI-SQUARES  3. 6026 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


3  OE6REES  OF  FREEDOMS 


7.6167 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 1763070*00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS* 


TAU 

OBS 

EXPECTED! CONDI 

SOI  CONDI 

3.00 

5 

3.65 

1.91 

2.00 

0 

2.01 

1.42 

2.00 

1 

1.69 

1.30 

5.00 

3 

1.38 

1.17 

3.00 

0 

-0.36 

0.60 

CHI-SQUAREsO .4353990*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


3  DEGREES  OF  FREEDOM* 


7.B167 


BINOMIAL 


THE  ESTIMATE  OF  N*  9 

THE  ESTIMATE  OF  As  0.22490*00 


TAU 

OBS 

EXPECTED! CONDI 

SOICOND 

3.00 

5 

4.52 

2.13 

2.00 

0 

1.53 

1.24 

2.00 

1 

1.53 

1.24 

5.00 

3 

2.17 

1.47 

3.00 

0 

0.10 

0.32 

V  V  v  ■ 


*  » 


CHI-SQUARE*  2.1812 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM*  7.8167 


IBM  POISSON  HITH  VARIABLE  ALPHA 

APS  ASZ  IT 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K*0. 1024710*01 
NUMBER  OF  ITERATIONS*  6 

THE  ESTIMATE  OF  LAM8DA*0. 1004550*00 


TAU 

oes 

EXPECTED ( CONO J 

SOI CONO 

54.00 

3 

11.12 

3.34 

2.00 

'0 

0.76 

0.87 

4.00 

0 

1.64 

1.28 

5.00 

2 

2.28 

1.51 

2.00 

1 

0.99 

1.00 

1.00 

0 

0.52 

0.72 

1.00 

2 

0.53 

0.73 

1.00 

2 

0.54 

0.74 

3.00 

10 

1.71 

1.31 

1.00 

3 

0.60 

0.77 

1.00 

1 

0.61 

0.78 

5.00 

12 

3.29 

1.81 

2.00 

3 

1.43 

1.20 

1.00 

1 

0.74 

0.86 

2.00 

1 

1.54 

1.24 

2.00 

2 

1.62 

1.27 

2.00 

2 

1.70 

1.30 

2.00 

0 

1.79 

1.34 

3.00 

2 

2.85 

1.69 

3.00 

3 

3.06 

1.75 

1.00 

2 

1.07 

1.04 

1.00 

1 

1.10 

1.05 

1.00 

1 

1.13 

1.06 

6.00 

2 

7.36 

2.71 

5.00 

1 

7.02 

2.65 

CHI-SQUARE*0. 1042740*03 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  23  DEGREES  OF  FREEDOM*  35.1779 


JELINSKI-MORANOA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N*  -3 
THE  ESTIMATE  OF  B=-0 . 24410-01 
NUMBER  OF  ITERATIONS*  3 


TAU 


OBS 


EXPECTED! CONO) 


SD(COND) 


54.00 

3 

11.12 

3.34 

2.00 

0 

0.76 

0.87 

4.00 

0 

1.64 

1.28 

5.00 

2 

2.28 

1.51 

2.00 

1 

0.99 

1.00 

1.00 

0 

0.52 

0.72 

1.00 

2 

0.53 

0.73 

1.00 

2 

0.54 

0.74 

3.00 

10 

1.71 

1.31 

1.00 

3 

0.60 

0.77 

1.00 

1 

0.61 

0.78 

5.00 

12 

3.29 

1.81 

2.00 

3 

1.43 

1.20 

1.00 

1 

0.74 

0.86 

2.00 

1 

1.54 

1.24 

2.00 

2 

1.62 

1.27 

2.00 

2 

1.70 

1.30 

2.00 

0 

1.79 

1.34 

3.00 

2 

2.85 

1.69 

3.00 

3 

3.06 

1.75 

1.00 

2 

1.07 

1.04 

1.00 

1 

1.10 

1.05 

1.00 

1 

1.13 

1.06 

6.00 

2 

7.36 

2.71 

5.00 

1 

7.02 

2.65 

CHI-SQUAREs 

104.2743 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  23  DE6REES  OF  FREEDOMS  35.1779 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  258 
THE  ESTIMATE  OF  ALPHAS  0.26720*00 
THE  ESTIMATE  OF  PHIs  0.7691D-02 
NUMBER  OF  ITERATIONS3  12 


TAU 

OBS 

EXPECTED! CONO) 

SO! CONO) 

54.00 

3 

5.75 

2.40 

2.00 

0 

2.38 

1.54 

4.00 

0 

2.85 

1.69 

5.00 

2 

3.01 

1.74 

2.00 

1 

2.35 

1.53 

1.00 

0 

1.94 

1.39 

1.00 

2 

1.94 

1.39 

1.00 

2 

1.93 

1.39 

3.00 

10 

2.54 

1.60 

1.00 

3 

1.87 

1.37 

1.00 

1 

1.83 

1.35 

5.00 

12 

2.77 

1.67 

•5 


* 


2.00 

2 

2.00 

2.00 

0 

1.99 

3.00 

2 

2.18 

3.00 

3 

2.16 

1.00 

2 

1.60 

1.00 

1 

1.59 

1.00 

1 

1.58 

6.00 

2 

2.52 

5.00 

1 

2.38 

1.41 

1.41 

1.48 

1.47 

1.27 

1.26 

1.26 

1.59 

1.54 


THE  CIMUUTIVE  NUMBER  OF  ERRORS  OBSERVED=0. 5700000*02 


CHI-SQUARE*  68.1667 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  22  DEGREES  OF  FREEDOM*  33.9327 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*0. 2989160-04 
NUMBER  OF  ITERATIONS*  10 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  8996 


TAU 

OBS 

EXPECTED  (CONDI 

SOI COND 

54.00 

3 

27.77 

5.27 

2.00 

0 

1.03 

1.01 

4.00 

0 

2.06 

1.44 

5.00 

2 

2.57 

1.60 

2.00 

1 

1.03 

1.01 

1.00 

0 

0.51 

0.72 

1.00 

2 

0.51 

0.72 

1.00 

2 

0.51 

0.72 

3.00 

10 

1.54 

1.24 

1.00 

3 

0.51 

0.72 

1.00 

1 

0.51 

0.72 

5.00 

12 

2.57 

1.60 

2.00 

3 

1.03 

1.01 

1.00 

1 

0.51 

0.72 

2.00 

1 

1.03 

1.01 

2.00 

2 

1.03 

1.01 

2.00 

2 

1.03 

1.01 

2.00 

0 

1.03 

1.01 

3.00 

2 

1.54 

1.24 

3.00 

3 

1.54 

1.24 

1.00 

2 

0.51 

0.72 

1.00 

1 

0.51 

0.72 

1.00 

1 

0.51 

0.72 

6.00 

2 

3.07 

1.75 

5.00 

1 

2.56 

1.60 

CHI -SQUARE*Q. 1430810*03 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  23  DEGREES  OF  FREEDOM*  35.1779 


BINOMIAL 


THE  ESTIMATE  OF  N*  14 
THE  ESTIHATE  OF  A*  0. 21690-11 


TAU 

oes 

EXPECTED (CONDI 

S0CC0N0 

54.00 

3 

0.00 

0.00 

2.00 

0 

0.00 

0.00 

4.00 

0 

0.00 

0.00 

5.00 

2 

0.00 

0.00 

2.00 

1 

0.00 

0.00 

1.00 

0 

0.00 

0.00 

1.00 

2 

0.00 

0.00 

1.00 

2 

0.00 

0.00 

3.00 

10 

0.00 

0.00 

1.00 

3 

-0.00 

0.00 

1.00 

1 

-0.00 

0.00 

5.00 

12 

-0.00 

0.00 

2.00 

3 

-0.00 

0.00 

1.00 

1 

-0.00 

0.00 

2.00 

1 

-0.00 

0.00 

2.00 

2 

-0.00 

0.00 

2.00 

2 

-0.00 

0.00 

2.00 

0 

-0.00 

0.00 

3.00 

2 

-0.00 

0.00 

3.00 

3 

-0.00 

0.00 

1.00 

2 

-0.00 

0.00 

1.00 

1 

-0.00 

0.00 

1.00 

1 

-0.00 

0.00 

6.00 

2 

-0.00 

0.00 

5.00 

1 

-0.00 

0.00 

CHI -SQUARE -*****«*»*»« 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH  23  DEGREES  OF  FREEDOMS 


35.1779 


ASZ  SO 


IBM  POISSON  HITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


THE  ESTIHATE  OF  K*0. 9324640*00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIHATE  OF  LAM80A*0. 1462700*01 


TAU 

OBS 

EXPECTED! C0ND) 

SO1C0ND 

1.00 

1 

1.46 

1.21 

3.00 

3 

3.62 

1.95 

2.00 

5 

2.14 

1.46 

1.00 

0 

0.96 

0.96 

2.00 

2 

1.73 

1.32 

4.00 

2 

2.62 

1.68 

2.00 

1 

1.14 

1.07 

3.00 

2 

1.44 

1.20 

.%  .vy  .'vVv  /vV.*  ". 
o' . 

to  »  to  *  *N  » '  »  ’*  •  •  •  -  •  «  •  • 
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•  v* 
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CHI-SQUARE*0 . 7014240*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOMS 


JELINSKI-MORANDA 


THE  ESTIMATE  OF  N*  22 
THE  ESTIMATE  OF  ALPHA=  0.10000*01 
THE  ESTIMATE  OF  PHI*  0.57260-01 
HUMBER  OF  ITERATIONS*  10 


TAU  OBS 

EXPECTED(COND) 

SO (CONG 

1.00 

1 

1.28 

1.13 

3.00 

3 

3.67 

1.92 

2.00 

5 

2.33 

1.53 

1.00 

0 

0.99 

1.00 

2.00 

2 

1.76 

1.33 

4.00 

2 

2.60 

1.61 

2.00 

1 

1.19 

1.09 

3.00 

2 

1.61 

1.27 

3.00 

0 

1.26 

1.12 

5.00 

2 

1.53 

1.24 

4.00 

1 

0.77 

0.88 

CHI-SQUARE* 

6. 

0026 

0.950  QUANTILE 

FOR 

CHI-SQUARE  WITH 

9  DEGREES  ( 

NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  22 
THE  ESTIMATE  OF  B=  0.69920-01 
NUMBER  OF  ITERATIONS*  2 


TAU 

OBS 

EXPECTED (CONDI 

SO  (COM3 

1.00 

1 

1.46 

1.21 

3.00 

3 

3.82 

1.95 

2.00 

5 

2.14 

1.46 

1.00 

0 

0.96 

0.98 

2.00 

2 

1.73 

1.32 

4.00 

2 

2.82 

1.68 

2.00 

1 

1.14 

1.07 

3.00 

2 

1.44 

1.20 

3.00 

0 

1.16 

1.08 

5.00 

2 

1.47 

1.21 

4.00 

1 

0.86 

0.93 

CHI-SQUARE*  7.0142 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEOOM* 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  Ns  22 
THE  ESTIMATE  OF  ALPHA*  0. 11460*01 
THE  ESTIMATE  OF  PHI*  0.51650-01 
NUMBER  OF  ITERATIONS*  6 


TAU 

OBS 

EXPECTED I CONO) 

SO (CONO 

1.00 

1 

1.14 

1.07 

3.00 

3 

3.63 

1.96 

2.00 

5 

2.29 

1.51 

1.00 

0 

0.66 

0.94 

2.00 

2 

1.71 

1.31 

4.00 

2 

2.78 

1.67 

2.00 

1 

1.14 

1.07 

3.00 

2 

1.63 

1.26 

3.00 

0 

1.26 

1.12 

5.00 

2 

1.61 

1.27 

4.00 

1 

0.74 

0.66 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVEO*0. 1900000*02 


CHI-SQUARE*  6.1032 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM* 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 5658460-01 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  23 


TAU 

OBS 

EXPECTED ( CONO ) 

SDtCOND 

1.00 

1 

1.33 

1.15 

3.00 

3 

3.59 

1.90 

2.00 

5 

2.35 

1.53 

1.00 

0 

1.04 

1.02 

2.00 

2 

1.76 

1.34 

4.00 

2 

2.51 

1.58 

2.00 

1 

1.22 

1.10 

3.00 

2 

1.61 

1.27 

3.00 

0 

1.27 

1.13 

5.00 

2 

1.48 

1.22 

4.00 

1 

0.79 

0.89 

CHI-SQUARE*0 . 5966180*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOM* 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  A*  0.74100-01 


TAU 

OBS 

EXPECTED (COND) 

SO(CONO) 

1.00 

1 

1.51 

1.23 

3.00 

3 

4.03 

2.01 

2.00 

5 

2.37 

1.54 

1.00 

0 

0.87 

0.93 

2.00 

2 

1.68 

1.30 

4.00 

2 

2.62 

1.02 

2.00 

1 

1.13 

1.00 

3.00 

2 

1.43 

1.20 

3.00 

0 

1.04 

1.02 

5.00 

2 

1.01 

1.27 

4.00 

1 

0.82 

0.91 

CHI-SQUARE* 

5.8420 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DECREES  OF  FREEDOM*  10.9252 


APS  ASZ  ST 


IBM  POISSON  WITH  VARIABLE  ALPHA 


6E0METRIC  POISSON 


THE  ESTIMATE  OF  K=0. 1021170*01 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMBDA=0. 406 7900*00 


TAU 

OBS 

EXPECTED (COND) 

SD(COND) 

1.00 

0 

0.41 

0.64 

7.00 

3 

3.10 

1.70 

5.00 

3 

2.51 

1.58 

3.00 

3 

1.64 

1.28 

4.00 

1 

2.35 

1.53 

CHI-SQUARE*0. 2410130*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


3  DECREES  OF  FREEDOM* 


7.8107 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  -18 
THE  ESTIMATE  OF  B=-0. 20950-01 
NUMBER  OF  ITERATIONS*  2 


,  **. 


k 


V  V 


TAU  OBS 

EXPECTED (CONO) 

SO(COND) 

1.00  0 

0.41 

0.64 

7.00  3 

3.10 

1.76 

5.00  3 

2.51 

1.56 

3.00  3 

1.64 

1.26 

4.00  1 

2.35 

1.53 

CHI-SQUARE=  2. 

4161 

0.950  QUANTILE  FOR 

CHI-SQUARE  WITH 

3  DEGREES  OF 

GENERALIZED  POISSON 

IBM 

POISSON 

THE  ESTIMATE  OF  PHI=0. 5061910-04 

NUMBER  OF  ITERATIONS2  10 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS2  5643 

TAU  OBS 

EXPECTED (CONO) 

SOI  CONO) 

1.00  0 

0.50 

0.71 

7.00  3 

3.50 

1.67 

5.00  3 

2.50 

1.56 

3.00  3 

1.50 

1.22 

4.00  1 

2.00 

1.41 

CHI-SQUARE=0 . 2672470*01 

0.950  QUANTILE  FOR 

CHI-SQUARE  WITH 

3  0E6REES  OF 

BINOMIAL 

THE  ESTIMATE  OF  N= 

3 

THE  ESTIMATE  OF  A= 

0.63260-12 

TAU  OBS 

EXPECTED (CONO) 

SO(CONO) 

1.00  0 

0.00 

0.00 

7.00  3 

0.00 

0.00 

5.00  3 

-0.00 

0.00 

3.00  3 

-0.00 

0.00 

4.00  1 

-0.00 

0.00 

7.6167 


7.6167 


CHI-SQUARE2*********** 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


5  0E6REES  OF  FREEDOM2 

IBH  POISSON  WITH  VARIABLE  ALPHA 


7.6167 


APS 

APS 


ASZ  IN 
AC Z  IT 


A-22 


s*. 


L  *  '  »- 


•  • 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 97794 90*00 
NUMBER  OF  ITERATIONS5  4 

THE  ESTIMATE  OF  LAMBOA50. 6332540*00 


TAU 

OBS 

EXPECTED!  COM)) 

SO ( COND 

1.00 

0 

0.63 

0.80 

17.00 

3 

8.86 

2.98 

3.00 

5 

1.24 

1.12 

3.00 

2 

1.16 

1.08 

1.00 

0 

0.37 

0.61 

3.00 

2 

1.06 

1.03 

2.00 

2 

0.67 

0.82 

2.00 

0 

0.64 

0.80 

6.00 

3 

1.76 

1.33 

3.00 

2 

0.80 

0.89 

1.00 

0 

0.25 

0.50 

6.00 

4 

1.41 

1.19 

5.00 

1 

1.04 

1.02 

4.00 

0 

0.75 

0.87 

6.00 

0 

1.01 

1.00 

18.00 

0 

2.33 

1.53 

CHI-SQUARE=0. 327X690*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  14  DEGRCES  OF  FREEDOM 


JELINSKI-MORANOA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  29 
THE  ESTIMATE  OF  B=  0.2230D-01 
NUMBER  OF  ITERATIONS5  2 


TAU  OBS  EXPECTEO ( COND )  SD(CONO) 


4.00 

6.00 

18.00 


0 

0 

0 


0.75 

1.01 

2.33 


0.87 

1.00 

1.53 


CHI-SQUARE*  32.7169 

0.950  QUANTILE  FOR  CHI-SQUARE  MXTH  14  0E6REES  OF  FREEDOM*  23.6908 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  20 
THE  ESTIMATE  OF  ALPHA*  0.61320*00 
THE  ESTIMATE  OF  PHI*  0.10200*00 
NUMBER  OF  ITERATIONS*  7 


TAU  OBS  EXPECTED! CONO )  SO(CONO) 


1.00 

0 

2.01 

1.42 

17.00 

3 

10.86 

3.30 

3.00 

5 

3.35 

1.83 

3.00 

2 

3.15 

1.77 

1.00 

0 

1.50 

1.23 

3.00 

2 

2.35 

1.53 

2.00 

2 

1.21 

1.10 

2.00 

0 

1.05 

1.02 

6.00 

3 

1.75 

1.32 

3.00 

2 

0.55 

0.74 

1.00 

0 

0.18 

0.42 

6.00 

4 

0.22 

0.47 

5.00 

1 

-0.07 

0.27 

4.00 

0 

-0.54 

0.74 

6.00 

0 

-1.00 

1.00 

18.00 

0 

-2.56 

1.60 

THE  CUMULATIVE  NUMBER  OF  ERRORS  GBSERVEQsQ. 2400000*02 


CHI-SQUARE*  60.9782 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OP  FREEDOM*  EE. 3668 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*-. 4701820-04 
NUMBER  OF  ITERATIONS*  9 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  -3386 


TAU 

OBS 

EXPECTED (CONDI 

30(C0ND 

1.00 

0 

0.29 

0.54 

17.00 

3 

5.02 

2.24 

3.00 

5 

0.89 

0.94 

3.00 

2 

0.89 

0.94 

1.00 

0 

0.30 

0.54 

3.00 

2 

0.89 

0.94 

A-24 


r -■ 


CHX -SQUARED  .4103180*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  14  DEGREES  OF  FREEDOM: 


23.6900 


BINOMIAL 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  ACZ  SO 
APS  ACZ  ST 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  ICO. 1035450+01 
NUMBER  OF  ITERATIONS:  6 

THE  ESTIMATE  OF  LAMB0A=0. 1919020*00 


TAU 

OBS 

EXPECTED (COND) 

SO  (COND 

1.00 

0 

0.19 

0.44 

7.00 

1 

1.55 

1.24 

6.00 

2 

1.67 

1.29 

3.00 

2 

0.97 

0.99 

4.00 

2 

1.47 

1.21 

5.00 

1 

2.15 

1.47 

CHI-SQUARE=0. 2341600*01 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH 


4  DEGREES  OF  FREEDOM: 


9.4917 


JE  LINSKI -MORAND A 


N0NH0M0GENE0U3  POISSON 


THE  ESTIMATE  OF  N=  -4 
THE  ESTIMATE  OF  B=-0. 35030-01 
NUMBER  OF  ITERATIONS:  3 


TAU  OBS 


EXPECTED! COND ) 


SD(COND) 


7.00 

1 

1.55 

1.24 

6.00 

2 

1.67 

1.29 

3.00 

2 

0.97 

0.99 

4.00 

2 

1.47 

1.21 

5.00 

1 

2.15 

1.47 

CHI-SQUARE* 

2.3417 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM* 

6ENERALIZE0  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0 .3022690-04 
NUMBER  OF  ITERATIONS*  XO 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  5476 


TAU 

OBS 

EXPECTED! COND) 

SO! COMO 

1.00 

0 

0.31 

0.55 

7.00 

1 

2.15 

1.47 

6.00 

2 

1.65 

1.36 

3.00 

2 

0.92 

0.96 

4.00 

2 

1.23 

1.11 

5.00 

1 

1.54 

1.24 

CHI-SQUARE=0 . 2664930*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM* 


BINOMIAL 


THE  ESTIMATE  OF  N=  3 

THE  ESTIMATE  OF  A=-0. 22640-14 


TAU 

OBS 

EXPECTED! COND) 

SDICOND 

1.00 

0 

-0.00 

0.00 

7.00 

1 

-0.00 

0.00 

6.00 

2 

-0.00 

0.00 

3.00 

2 

0.00 

0.00 

4.00 

2 

0.00 

0.00 

5.00 

1 

0.00 

0.00 

CHI-3QUARE*»»***«hhhh»* 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM1 


9.4917 


9.4917 


9.4917 


APS  ACZ  IN 
APS  MMC  IT 


IBM  POISSON  WITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 1005300+01 
NUMBER  OF  ITERATIONS3  4 

THE  ESTIMATE  OF  LAMBDA30. 1146990*00 


TAU 

OBS 

EXPECTED! COND ) 

SO! COND 

8.00 

1 

0.93 

0.97 

11.00 

0 

1.35 

1.16 

8.00 

2 

1.03 

1.02 

7.00 

1 

0.94 

0.97 

1.00 

1 

0.14 

0.37 

5.00 

2 

0.70 

0.84 

34.00 

2 

5.26 

2.29 

4.00 

2 

0.68 

0.63 

5.00 

1 

0.88 

0.94 

6.00 

1 

1.08 

1.04 

CHI-SQUARE=0 . 1469960+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


JELINSKI-MORANOA 


a  DEGREES  OF  FREEDOM* 


15.5116 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  -21 
THE  ESTIMATE  OF  B3-0 .52850-02 
NUMBER  OF  ITERATIONS3  3 


TAU 

OBS 

EXPECTED! COND) 

SD(CONO) 

8.00 

1 

0.93 

0.97 

11.00 

0 

1.35 

1.16 

8.00 

2 

1.03 

1.02 

7.00 

1 

0.94 

0.97 

1.00 

1 

0.14 

0.37 

5.00 

2 

0.70 

0.84 

34.00 

2 

5.26 

2.29 

4.00 

2 

0.68 

0.83 

5.00 

1 

0.88 

0.94 

6.00 

1 

1.06 

1.04 

CHI-SQUARE3  14.6996 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


8  DEGREES  OF  FREEDOM* 


15.5116 


*1 


GENERALIZED  POISSON 


IBM  POISSON 


y-v. 


.v. 


the  ESTIMATE  OF  PHI sO . 1041 020-02 
NUMBER  of  ITERATIONS'  10 

THE  E5TIMATE  of  THE  NUMBER  OF  INITIAL  BUSS*  145 


1 

t.y. 


TAU  OBS 

EXPECTEOI  CONDI 

SD  1  CON’D 

8.00  1 

1.23 

1.11 

11.00  o 

1.68 

1.30 

8.00  2 

1.21 

1.10 

7.00  1 

1.05 

1.02 

1.00  1 

0.15 

0.39 

5.00  2 

0.74 

0.86 

34.00  2 

4.87 

2.21 

4.00  2 

0.58 

0.76 

5.00  1 

0.71 

0.84 

6.00  1 

0.84 

0.92 

CHI-SQUAREsO. 1459210*02 

0.950  QUANTILE  FOR  CHI- 

-SQUARE  WITH  8  DEGREES  0 

BINOMIAL 

IBM  POISSON  WITH  VARIABLE 

ALPHA 

APS  MMC  SD 

GEOMETRIC 

:  POISSON 

THE  ESTIMATE  OF  K=0 . 9259380*00 

NUMBER  OF  ITERATIONS* 

4 

THE  ESTIMATE  OF  LAMBDA* 

0.3951600*00 

TAU  OBS  EXPECTEOt  COMO ) 

SO  (CONDI 

1.00  1 

0.40 

0.63 

4.00  0 

1.31 

1.14 

5.00  2 

1.16 

l.OS 

6.00  1 

0.91 

0.96 

2.00  0 

0.22 

0.47 

15.5118 


:-S 


CHI-SQUAREsO . 3073590*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


JELINSK.I-MORANOA 


3  DEGREES  CF  FREEDOM* 


7.8167 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N= 


IP 


A-28 


THE  ESTIMATE  OF  B=  0.7695D-01 
NUMBER  OF  ITERATIONS:  2 


TAU  OBS 


EXPECTED (COND)  SD(COND) 


1.00 

1 

0.40 

4.00 

0 

1.31 

5.00 

2 

1.16 

6.00 

1 

0.91 

2.00 

0 

0.22 

0.63 

1.14 

1.08 

0.96 

0.47 


CHI-SQUARE:  3.0736 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


3  DEGREES  OF  FREEDOM 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*-. 4134880-04 
NUMBER  OF  ITERATIONS*  14 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS=  -2742 


TAU 

OBS 

EXPECTED (COND) 

SDtCOND 

1.00 

1 

0.22 

0.47 

4.00 

0 

0.89 

0.94 

5.00 

2 

1.11 

1.05 

6.00 

1 

1.33 

1.15 

2.00 

0 

0.44 

0.67 

CHI-SQUARE=0. 4854320*01 

0.950  QUANTILE  FOR  CHI-SQUARE  NXTH  3  0E6REES  OF  FREEDOM 


BINOMIAL 


THE  ESTIMATE  OF  N=  5 

THE  ESTIMATE  OF  A*  0.69890-01 


TAU 

OBS 

EXPECTED (COND) 

SD(CONO 

1.00 

1 

0.37 

0.61 

4.00 

0 

1.09 

1.05 

5.00 

2 

1.32 

1.15 

6.00 

1 

0.85 

0.92 

2.00 

0 

0.19 

0.44 

CHI-SQUARE*  2.7291 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH  3  DEGREES  OF  FREEDOM 


IBM  POISSON  HITH  VARIABLE  ALPHA 


APS  MMC  ST 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0 .9228650*00 
NUMBER  OF  ITERATIONS:  4 

THE  ESTIMATE  OF  UM80A=0. 1312110*01 


TAU 

OBS 

EXPECTED! CONO) 

SO (CONO 

3.00 

2 

3.64 

1.91 

4.00 

6 

3.67 

1.92 

5.00 

3 

3.21 

1.79 

6.00 

2 

2.48 

1.58 

CHI -SQUARE=0. 2321870*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM: 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  17 
THE  ESTIMATE  OF  B=  0.80270-01 
NUMBER  OF  ITERATIONS:  2 


TAU  OBS  EXPECTED (CONO)  SO(CONO) 


3.00 

2 

3.64 

4.00 

6 

3.67 

5.00 

3 

3.21 

6.00 

2 

2.48 

1.91 

1.92 
1.79 
1.58 


CHI-SQUARE=  2.3219 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM: 


GENERALIZED  POISSON 


IBM  POISSON 

THE  ESTIMATE  OF  PHI:0. 7834570-01 
NUMBER  OF  ITERATIONS:  3 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS:  16 


TAU  OBS 


EXPECTED ( COffl) )  SD ( COND  ) 


A-30 


5.9948 


5.9948 


CHI-5QUARE-0 . 1386740*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  0E6REES  OF  FREEDOM2 


BINOMIAL 


THE 

ESTIMATE 

OF 

N= 

17 

THE 

ESTIMATE 

OF 

A2 

0.84680-01 

TAU 

OBS 

EXPECTED (COND) 

SO(CONO) 

3.00 

2 

3.75 

1.94 

4.00 

6 

4.23 

2.06 

5.00 

3 

3.01 

1.74 

6.00 

2 

2.28 

1.51 

CHI-SQUARE2 

1.5927 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM2 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  MMC  IN 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9878610*00 
NUMBER  OF  ITERATIONS2  5 

THE  ESTIMATE  OF  LAMBOA=0 . 1885420*01 


TAU 

OBS 

EXPECTED! COND  1 

SO! COND 

5.00 

8 

9.20 

3.03 

1.00 

2 

1.77 

1.33 

4.00 

9 

6.88 

2.62 

4.00 

6 

6.55 

2.56 

1.00 

1 

1.59 

1.26 

CHI-SQUARE20 . 1102550*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM2 


JELINSKI-MORANOA 


NONHOMOGENEOUS  POISSON 


5.9948 


5. 9948 


7.8167 


THE  ESTIMATE  OF  N= 


155 


THE  ESTIMATE  OF  8*  0.12210-01 
NUMBER  OF  ITERATIONS9  2 


TAU  DBS 


EXPECTED!  CONDI  SOI  CONDI 


i 


I 


5.00 

8 

9.20 

3.03 

1.00 

2 

1.77 

1.33 

4.00 

9 

6.88 

2.62 

4.00 

6 

6.55 

2.56 

1.00 

1 

1.59 

1.26 

CHI-SQUARE9  1.1025 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM’ 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  69 
THE  ESTIMATE  OF  ALPHA9  0.97920*00 
THE  ESTIMATE  OF  PHI9  0.28730-01 
NUMBER  OF  ITERATIONS9  9 


TAU 

OBS 

EXPECTED (CONDI 

SO  (COM3 

5.00 

8 

9.62 

3.10 

1.00 

2 

1.96 

1.40 

4.00 

9 

7.17 

2.68 

4.00 

6 

5.95 

2.44 

1.00 

1 

1.30 

1.14 

THE  CUMULATIVE  NUM8ER  OF  ERRORS  0BSERVED90. 2600000*02 


CHI-SQUARE9  0.8083 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM9 


IBM  POISSON 


THE  ESTIMATE  OF  PHI90. 6025540-04 
NUMBER  OF  ITERATIONS9  8 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS9  20401 


TAU 

OBS 

EXPECTED (CONDI 

SOI  COM) 

5.00 

8 

8.67 

2.94 

1.00 

2 

1.73 

1.32 

4.00 

9 

6.93 

2.63 

4.00 

6 

6.93 

2.63 

1.00 

1 

1.73 

1.32 

7.8167 


5.9948 


CHI-SQUARE90 .1142320*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  OEGREES  OF  FREEDOM9 


7.8167 


APS  ATZ  IT 


BINOMIAL 


IBM  POISSON  WITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 1001930m 
NUMBER  OF  ITERATIONS:  4 

THE  ESTIMATE  OF  LAMBDA=0. 1783760+00 


TAU 

OBS 

EXPECTED (COND) 

SO (COND 

28.00 

4 

5.13 

2.26 

14.00 

3 

2.67 

1.63 

1.00 

0 

0.19 

0.44 

4.00 

2 

0.78 

0.88 

2.00 

0 

0.39 

0.63 

5.00 

1 

0.98 

0.99 

3.00 

2 

0.60 

0.77 

1.00 

1 

0.20 

0.45 

2.00 

1 

0.40 

0.63 

18.00 

1 

3.66 

1.91 

CHI-SQUAREsO. 1217560*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


JELINSKI-MORANDA 


B  0E6REES  OF  FREEDOM: 


15.5118 


N0NH0MQSENE0U3  POISSON 


THE  ESTIMATE  OF  N=  -91 
THE  ESTIMATE  OF  B=-0. 19290-02 
NUMBER  OF  ITERATIONS:  3 


TAU 

OBS 

EXPECTED (CONO) 

SD(CONO) 

28.00 

4 

5.13 

2.26 

14.00 

3 

2.67 

1.63 

1.00 

0 

0.19 

0.44 

APS  ATZ  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K:0. 1001930*01 
NUMBER  OF  ITERATIONS:  4 

THE  ESTIMATE  OF  LAM80A=0. 1763760*00 


*  .  *  4  ■ 


•  «  *^4  *  «  •  . 

■  ^  -  •  *  .  ’  4  4.  * 


.s 

*  -  ^  . 


TAU 

OBS 

EXPECTED! CONO ) 

28.00 

4 

5.13 

14.00 

3 

2.67 

1.00 

0 

0.19 

4.00 

2 

0.78 

2.00 

0 

0.39 

5.00 

1 

0.98 

3.00 

2 

0.60 

1.00 

1 

0.20 

2.00 

1 

0.40 

18.00 

1 

3.66 

CHI-SQUARE=0. 1217560*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


JELINSKI-MORANDA 


N0NH0M06ENE0US  POISSON 


THE  ESTIMATE  OF  N=  -91 
THE  ESTIMATE  OF  B=-0 .19290-02 
NUMBER  OF  ITERATIONS:  3 


TAU 

OBS 

EXPECTED (COND ) 

28.00 

4 

5.13 

14.00 

3 

2.67 

1.00 

0 

0.19 

4.00 

2 

0.78 

2.00 

0 

0.39 

5.00 

1 

0.98 

3.00 

2 

0.60 

1.00 

1 

0.20 

2.00 

1 

0.40 

18.00 

1 

3.66 

CHI-SQUARE=  12.1756 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


6ENERALIZE0  POISSON 


THE  ESTIMATE  OF  N=  29 
THE  ESTIMATE  OF  ALPHA*  0.41350*00 
THE  ESTIMATE  OF  PHI*  0.3511D-01 
NUMBER  OF  ITERATIONS*  7 


TAU  OBS  EXPECTED  I COND) 

28.00  4  4.09 


SO (COND ) 

2.26 

1.63 

0.44 

0.88 

0.63 

0.99 

0.77 

0.45 

0.63 

1.91 


8  DECREES  OF  FREEDOM*  15.5118 


SO  (COND) 

2.26 

1.63 

0.44 

0.83 

0.63 

0.99 

0.77 

0.45 

0.63 

1.91 


8  DEGREES  OF  FREEDOM*  15.5118 


SD(CONO) 

2.02 


A-34 


14.00 

3 

2.55 

1.60 

1.00 

0 

0.79 

0.89 

4.00 

2 

1.33 

1.15 

2.00 

0 

0.95 

0.98 

5.00 

1 

1.32 

1.15 

3.00 

2 

1.02 

1.01 

1.00 

1 

0.57 

0.76 

2.00 

1 

0.72 

0.65 

18.00 

1 

1.67 

1.29 

THE  CUMULATIVE 

NUMBER 

OF  ERRORS  OBSERVED:Q . 150000D+02 

CHI-SQUARE= 
0.950  QUANTILE 

3.6823 
FOR  CHI 

-SQUARE  WITH 

7  DEGREES  OF  FREEDOM: 

14.0702 

IBM  POISSON 


THE  ESTIMATE  OF  PHI*0. 1162220-02 
NUMBER  OF  ITERATIONS:  10 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS:  173 


TAU 

OBS 

EXPECTED (CONO I 

SOICOND 

28.00 

4 

5.58 

2.36 

14.00 

3 

2.73 

1.65 

1.00 

0 

0.19 

0.44 

4.00 

2 

0.77 

o.ea 

2.00 

0 

0.38 

0.62 

5.00 

1 

0.95 

0.98 

3.00 

2 

0.57 

0.75 

1.00 

1 

0.19 

0.43 

2.00 

1 

0.37 

0.61 

18.00 

1 

3.30 

1.62 

CHI-3QUARE=0. 1280450+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  8  DEGREES  OF  FREEDOM:  15.5116 


BINOMIAL 


THE  ESTIMATE  OF  N:  30 

THE  ESTIMATE  OF  A=  0.59950-02 


TAU 


OBS 


EXPECTED! CONO) 


SD(  CONDI 


2.00 

10.00 


1 

1 


0.20 

1.63 


0.45 

1.28 


CHI-SQUARE*  23.5173 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  A  0E6REES  OF  FREEDOM*  15.5116 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  ATZ  SO 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 1116620+01 
NUMBER  OF  ITERATIONS*  6 

THE  ESTIMATE  OF  LAMBOA=0. 2534230*00 


TAU 

OBS 

EXPECTED! CONO) 

SOCCOND 

6.00 

1 

2.04 

1.43 

3.00 

2 

1.65 

1.29 

2.00 

2 

1.45 

1.20 

2.00 

4 

1.60 

1.34 

4.00 

3 

5.06 

2.25 

CHI-SQUAPE=0. 4319540*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM*  7.6167 


JELINSKI-MORANOA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  -1 
THE  ESTIMATE  OF  B=-0. 11030*00 
NUMBER  OF  ITERATIONS*  4 


TAU  OBS  EXPECTED (CONO)  SO(COND) 

6.00  1  2.04  1.43 

3.00  2  1.65  1.29 

2.00  2  1.45  1.20 

2.00  4  1.60  1.34 

4.00  3  5.06  2.25 

CHI-SQUARE*  4.3195 

0.950  QUANTILE  FOR  CHI-SQUARE  KITH  3  0E6RECS  OF  FREEDOM*  7.6167 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 4693690-04 
NUMBER  OF  ITERATIONS:  9 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUSS:  7666 


EXPECTED (COND)  SD(CONO) 


TAU  OBS 


6.00  1  4.24 
3.00  2  2.12 
2.00  2  1.41 
2.00  4  1.41 
4.00  3  2.82 


CHI-SQUARE=0 . 7479400+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 

BINOMIAL 


THE  ESTIMATE  OF  N= 
THE  ESTIMATE  OF  A= 


TAU  CBS 

6.00  1 

3.00  2 

2.00  2 

2.00  4 

4.00  3 


5 

0.18320-13 


EXPECTED (COM3) 

0.00 

0.00 

0.00 

-0.00 

-0.00 


2.06 

1.46 

1.19 

1.19 

1.68 


3  DEGREES  OF  FREEDOM 


SD(COND) 

0.00 

0.00 

0.00 

0.00 

0.00 


3  DEGREES  OF  FREEDOM: 


CHI -SQUARE =****«****«• 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  ATZ  ST 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0 .8712960+00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMB0A:0.788882D+00 


TAU 


OBS 


EXPECTEO(COND)  SO (COND) 


CHI-SQUARE=0. 5458860*00 

0.950  QUANTILE  FOR  CHI-SQUARE  NITH  3  DEGREES  OF  FREEDOM: 


JELINSKI-MORANOA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE 

OF  N= 

6 

THE  ESTIMATE 

OF  B:  0. 

13760*00 

NUMBER  OF  ITERATIONS: 

3 

TAU  OBS 

EXPECTED! CONDI 

SO(  CONDI 

1.00 

1 

0.79 

0.69 

3.00 

2 

1.81 

1.34 

1.00 

0 

0.45 

0.67 

8.00 

2 

2.06 

1.43 

15.00 

1 

0.69 

0.94 

CHI-SQUARE:  0.5459 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM: 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  7 

THE  ESTIMATE  OF  ALPHA:  0.10310*01 
THE  ESTIMATE  OF  PHI:  0.9970D-01 
NUMBER  OF  ITERATIONS:  12 


TAU 

OBS 

EXPECTED! CONDI 

SOICONO 

1.00 

1 

0.66 

0.81 

3.00 

2 

1.73 

1.32 

1.00 

0 

0.46 

0.68 

8.00 

2 

2.20 

1.48 

15.00 

1 

0.95 

0.96 

THE  CUMULATIVE  NUMBER  OF  ERRORS  0BSERVED=0. 6000000*01 

CHI-SQUARE:  0.6998 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM: 


IBM  POISSON 


THE  ESTIMATE  OF  PHI:0. 1035840*00 
NUMBER  OF  ITERATIONS:  6 


7.8167 


7.8167 


5.9948 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS: 


7 


TAU 

OBS 

EXPECTED (COM)! 

SDICOND 

1.00 

1 

0.75 

0.67 

3.00 

2 

1.75 

1.32 

1.00 

0 

0.54 

0.74 

8.00 

2 

1.89 

1.36 

15.00 

1 

1.01 

1.00 

CHI-SQUARE=0. 6689560*00 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DESREES  OF  FREEOOM= 


BINOMIAL 


THE  ESTIMATE 

OF  N= 

6 

THE  ESTIMATE 

OF  A= 

0.15640*00 

TAU 

OBS 

EXPECTEO ( COND ) 

SO (COND 

1.00 

1 

0.67 

0.93 

3.00 

2 

1.87 

1.37 

1.00 

0 

0.43 

0.66 

6-00 

2 

2.14 

1.46 

15.00 

1 

0.90 

0.95 

CHI-SQUARE-  0.4629 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOM 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  ATZ  IN 
APS  AAZ  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9641340*00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBDA=0. 7323660*00 


TAU 


CBS 


EXPECTED (CONDI 


SO  (CONDI 


CHI -SQUARE =0.848494D*01 

0.950  QUANTILE  FOR  CHI-SQUARE  UITH  9  DEGREES  OF  FREEDOM*  16.9252 


JELINSKI-MORANOA 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  ALPHA*  0.10000*01 
THE  ESTIMATE  OF  PHI*  0.34760-01 
NUMBER  OF  ITERATIONS*  7 


TAU 

OBS 

EXPECTED (CONDI 

30<  CONDI 

3.00 

1 

2.15 

1.47 

3.00 

3 

2.05 

1.43 

3.00 

1 

1.73 

1.32 

8.00 

2 

4.07 

2.02 

5.00 

4 

2.19 

1.48 

1.00 

0 

0.40 

0.64 

9.00 

3 

2.70 

1.64 

3.00 

1 

0.69 

0.83 

7.00 

2 

1.13 

1.06 

2.00 

X 

0.25 

0.50 

29.00 

1 

1.64 

1.28 

CHI-SQUARE* 

7 

.6318 

0.950  QUANTILE  FOR  CHI-SQUARE  UITH  9  OEGREES  OF  FREEDOM*  16.9252 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  20 
THE  ESTIMATE  OF  B=  0. 36530-01 
NUMBER  OF  ITERATIONS*  2 


TAU 

OBS 

EXPECTED C CONDI 

SO (COND 

3.00 

1 

2.12 

1.46 

3.00 

3 

1.90 

1.38 

3.00 

1 

1.70 

1.30 

8.00 

2 

3.72 

1.93 

5.00 

4 

1.63 

1.35 

1.00 

0 

0.33 

0.57 

9.00 

3 

2.47 

1.57 

3.00 

1 

0.66 

0.61 

7.00 

2 

1.26 

1.13 

2.00 

1 

0.31 

0.56 

29.00 

1 

2.67 

1.64 

CHI-SQUARE*  8.4849 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOM*  16.9252 


GENERALIZED  POISSON 


A-40 


THE  ESTIMATE  OF  N=  25 
THE  ESTIMATE  OF  ALPHA*  0.58450*00 
THE  ESTIMATE  OF  PHI*  0.4765D-01 
NUMBER  OF  ITERATIONS*  10 


TAU 

OBS 

EXPECTED! COHO) 

SD(CONO 

3.00 

1 

2.24 

1.50 

3.00 

3 

2.15 

1.47 

3.00 

1 

1.88 

1.37 

8.00 

2 

3.01 

1.73 

5.00 

4 

2.04 

1.43 

1.00 

0 

0.75 

0.87 

9.00 

3 

2.19 

1.48 

3.00 

1 

0.97 

0.98 

7.00 

2 

1.29 

1.14 

2.00 

1 

0.55 

0.74 

29.00 

1 

1.95 

1.40 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED*0. 1900000*02 


CHI-SQUARE*  5.9170 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  8  DEGREES  OF  FREEOOM* 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 34894 30-01 
NUMBER  OF  ITERATIONS*  4 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  22 


TAU 

08S 

EXPECTED! COND) 

SO! COND 

3.00 

1 

2.19 

1.48 

3.00 

3 

Z  .09 

1.44 

3.00 

1 

1.78 

1.34 

8.00 

2 

3.87 

1.97 

5.00 

4 

2.22 

1.49 

1.00 

0 

0.44 

0.66 

9.00 

3 

2.64 

1.63 

3.00 

1 

0.77 

0.88 

7.00 

2 

1.25 

1.12 

2.00 

1 

0.32 

0.57 

29.00 

1 

1.71 

1.31 

CHI-SQUARE*0 .6475270*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOM* 


BINOMIAL 


THE  ESTIMATE  OF  N*  20 
THE  ESTIMATE  OF  A*  0.33400-01 


•*.  ■"J’.'-'r-  ■ |J-  *•  *r  *,•  ?  r.1 


rvrTTT^T* tt: 


TAU  OBS 

EXPECTEO ( COND ) 

3.00 

1 

1.94 

3.00 

3 

1.85 

3.00 

1 

1.56 

6.00 

2 

3.61 

5.00 

4 

2.06 

1.00 

0 

0.31 

9.00 

3 

2.44 

3.00 

1 

0.61 

7.00 

2 

1.12 

2.00 

1 

0.22 

29.00 

1 

1.48 

CHI-SQUARE: 

8. 

2558 

0.950  QUANTILE 

FOR 

CHI-SQUARE  WITH 

SO(CONO) 


1.39 

1.36 

1.25 

1.90 

1.43 

0.56 

1.56 

0.76 

1.06 

0.47 

1.22 


9  DEGREES  OF  FREEDOM:  16.9252 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  AAZ  SO 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9593920+00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMB0A=0. 4357940+00 


TAU 

OBS 

EXPECTED (COND) 

SDCCOND 

6.00 

1 

2.36 

1.54 

2.00 

0 

0.67 

0.82 

7.00 

3 

1.94 

1.39 

2.00 

2 

0.46 

0.68 

3.00 

2 

0.62 

0.79 

8.00 

0 

1.32 

1.15 

16.00 

1 

1.63 

1.28 

CHI-SQUARE=0. 1184620+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM:  11.0733 


JELINSKI-MORANDA 

THE  ESTIMATE  OF  N=  10 

THE  ESTIMATE  OF  ALPHA:  0.1000D+01 

THE  ESTIMATE  OF  PHI=  0.42850-01 


NUMBER  OF 

ITERATIONS: 

20 

TAU 

OBS 

EXPECTED (COND) 

SO(COND) 

6.00 

1 

2.68 

1.64 

2.00 

0 

0.81 

0.90 

to 


t"' 


k  '**  *•  *- 


k-k  2 


*  A  g*  ■  +  g  *  *  *  +  +  .  '  ■  •  .*#*«*  fc  •  «  ^  I  *  *  '*  #  *  g  ^  g  *  »  +  |  '  4  A  ^ «  1  ^  R  g  4  %  4  •  .***•**•*■  •* .  '  .•*+-' 


h  *• 


CHI-SQUARE*  9.3311 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM*  11.0733 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  11 
THE  ESTIMATE  OF  B*  0.41460-01 
NUMBER  OF  ITERATIONS*  2 


TAU 

OBS 

EXPECTED (COND) 

SO (COND 

6.00 

1 

2.36 

1.54 

2.00 

0 

0.67 

0.82 

7.00 

3 

1.94 

1.39 

2.00 

2 

0.46 

0.68 

3.00 

2 

0.62 

0.79 

8.00 

0 

1.32 

1.15 

16.00 

1 

1.63 

1.28 

CHI-SQUARE* 

11 

.8482 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM*  11.0733 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N*  16 
THE  ESTIMATE  OF  ALPHA*  0.17760*00 
THE  ESTIMATE  OF  FHI*  0. 84320-01 
NUMBER  OF  ITERATIONS*  8 


TAU 

OBS 

EXPECTED (COND) 

SDCCOND 

6.00 

1 

1.83 

1.35 

2.00 

0 

1.41 

1.19 

7.00 

3 

1.64 

1.28 

2.00 

2 

1.22 

1.10 

3.00 

2 

1.00 

1.00 

8.00 

0 

0.95 

0.98 

16.00 

1 

0.94 

0.97 

THE  CUMULATIVE  NUMBER  OF  ERRORS  0BSERVE0*0. 9000000*01 

CHI-SQUARE*  5.3483 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM*  9.4917 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 4672600-01 
NUMBER  OF  ITERATIONS®  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS®  11 


TAU 

OBS 

EXPECTED! COND ) 

SDICOND 

6.00 

1 

2.68 

1.64 

2.00 

0 

0.89 

0.94 

7.00 

3 

2.48 

1.58 

2.00 

2 

0.70 

0.84 

3.00 

2 

0.63 

0.79 

8.00 

0 

0.86 

0.93 

16.00 

1 

0.92 

0.96 

CHI-SQUARE®0. 8271550401 

0.550  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEOOM®  11.0733 


BINOMIAL 


THE  ESTIMATE  OF  N=  10 
THE  ESTIMATE  OF  A®  0.38120-01 


TAU 

OBS 

EXPECTED (COND) 

SDICOND 

6.00 

1 

2.11 

1.45 

2.00 

0 

0.69 

0.83 

7.00 

3 

2.19 

1.48 

2.00 

2 

0.46 

0.68 

3.00 

2 

0.47 

0.68 

8.00 

0 

0.61 

0.78 

16.00 

1 

1.07 

1.03 

CHI-SQUARE® 

12 

.2701 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM®  11.0733 


IBM  POISSON  MITH  VARIABLE  ALPHA 

APS  AAZ  ST 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  **0.9326820400 
NUMBER  OF  ITERATIONS®  5 

THE  ESTIMATE  OF  LAMBDA*0. 5021430400 


TAU 

OBS 

EXPECTED! COND) 

SDICOND) 

1.00 

1 

0.50 

0.71 

2.00 

1 

0.91 

0.95 

A -4  4 


CHI-SQUARE?0. 1402420+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM? 


12.5961 


BINOMIAL 


THE  ESTIMATE  OF  N=  7 

THE  ESTIMATE  OF  A=  0.7355D-01 


TAU 

OBS 

EXPECTED (COND) 

SD(COND 

1.00 

1 

0.50 

0.71 

2.00 

1 

0.83 

0.91 

4.00 

1 

1.30 

1.14 

6.00 

1 

1.83 

1.35 

4.00 

0 

0.79 

0.89 

5.00 

3 

0.96 

0.98 

6.00 

0 

0.04 

0.19 

10.00 

0 

0.05 

0.23 

CHI-SQUARE? 

6. 

.2246 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM?  12.5961 


IBM  POISSON  MITH  VARIABLE  ALPHA 

APS  AAZ  IN 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 6687820+00 
NUMBER  OF  ITERATIONS?  4 

THE  ESTIMATE  OF  LAMBQA?0. 1574040+01 


TAU 

OBS 

EXPECTED( COND ) 

SO (COND 

3.00 

3 

3.40 

1.85 

4.00 

2 

1.28 

1.13 

2.00 

0 

0.20 

0.44 

3.00 

0 

0.12 

0.34 

CHI-SQUARE=0 . 7664370+00 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM?  5.9948 


JELINSKI-MORANOA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  5 

THE  ESTIMATE  OF  B=  0. 3728D+00 


NUMBER  OF  ITERATIONS:  A 


TAU  OBS 


EXPECTED ( COND )  SDt  CONO > 


3.00 

3 

3.40 

4.00 

2 

1.28 

2.00 

0 

0.20 

3.00 

0 

0.12 

1.85 

1.13 

0.44 

0.34 


CHI-SQUARE:  0.7664 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM: 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0 .4252790-04 
NUMBER  OF  ITERATIONS:  9 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS:  4853 


TAU 

OBS 

EXPECTED (COND) 

SO (CONO 

3.00 

3 

1.25 

1.12 

4.00 

2 

1.67 

1.29 

2.00 

0 

0.83 

0.91 

3.00 

0 

1.25 

1.12 

CHI-SQUAREsO. 4596120*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DE6REES  OF  FREEDOM- 


BINOMIAL 


THE 

ESTIMATE 

OF 

N= 

5 

THE 

ESTIMATE 

OF 

A: 

0.29750*00 

TAU 

OBS 

EXPECTED (COND) 

SOI COND 

3.00 

3 

3.14 

1.77 

4.00 

2 

1.62 

1.27 

2.00 

0 

0.15 

0.38 

3.00 

0 

0.19 

0.44 

CHI-SQUARE: 

0.4358 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  2  DEGREES  OF  FREEDOM' 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  MEZ  IT 


5.9948 


5.9948 


5.9948 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0.977927D400 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBOA=0. 75306 70400 


TAU 

OBS 

EXPECTED ( COND 1 

14.00 

1 

9.16 

9.00 

8 

4.54 

2.00 

6 

0.89 

2.00 

2 

0.85 

1.00 

4 

0.41 

2.00 

0 

0.60 

5.00 

1 

1.84 

1.00 

0 

0.34 

7.00 

1 

2.21 

5.00 

1 

1.38 

2.00 

1 

0.51 

2.00 

1 

0.49 

11.00 

2 

2.33 

22.00 

1 

3.24 

CHI-SQUARE=0 . 7684640402 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SO(COND) 

3.03 
2. 13 
0.94 
0.92 
0.64 
0.69 
1.36 
0.59 
1.49 
1.17 
0.71 
0.70 
1.53 
1.60 


DEGREES  OF  FREEDOM:  21.0297 


JELINSKI-MORANDA 


THE  ESTIMATE  OF  N=  33 
THE  ESTIMATE  OF  ALPHA®  0.10000401 
THE  ESTIMATE  OF  PHI:  0.22380-01 
NUMBER  OF  ITERATIONS:  15 


TAU 

OBS 

EXPECTED (CONO) 

SD(COND) 

>> 

14.00 

1 

10.39 

3.22 

9.00 

8 

6.48 

2.55 

2.00 

6 

1.26 

1.12 

V  ,* 

2.00 

2 

1.22 

1.10 

1.00 

4 

0.41 

0.64 

w"  •  /  * 

2.00 

0 

0.77 

0.86 

mm 

5.00 

1 

1.36 

1.17 

1.00 

0 

0.25 

0.50 

7.00 

1 

1.59 

1.26 

5.00 

1 

1.03 

1.01 

.«<,r 

2.00 

2.00 

1 

1 

0.37 

0.32 

0.60 

0.57 

w  ..  * . 

11.00 

2 

1.52 

1.23 

22.00 

1 

2.05 

1.43 

tern 

CHI-SQUARE: 

63 

5094 

IV  V  0.950  QUANTILE  FOR  CHI-SQUARE  WITH  12  DEGREES  OF  FREEDOM: 


21.0297 


> «  •  .  ■ 


NONHOMOGENEOUS  POISSON 


A -A  8 


% 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 4277570-01 
NUMBER  OF  ITERATIONS:  7 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS=  28 


TAU 

OBS 

EXPECTEO(CONO) 

SO(CONO) 

14.00 

1 

12.87 

3.59 

- 

9.00 

8 

8.82 

2.97 

,  < 

2.00 

6 

1.94 

1.39 

2.00 

2 

1.85 

1.36 

1.00 

4 

0.56 

0.75 

2.00 

0 

1.02 

1.01 

5.00 

1 

1.40 

1.18 

1.00 

0 

0.26 

0.51 

•  ** 

7.00 

1 

1.36 

1.16 

Ej 

5.00 

1 

0.81 

0.90 

Pi 

2.00 

1 

0.26 

0.51 

2.00 

1 

0.18 

0.42 

11.00 

2 

0.44 

0.66 

1 

22.00 

1 

-0.53 

0.73 

CHI-SQUARE=0. 4910770*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  12  DEGPEES  OF  FREEDOM:  21.0297 


BINOMIAL 


THE  ESTIMATE  OF  N=  12 
THE  ESTIMATE  OF  A*  0.67210-11 


TAU 

OBS 

EXPECTED (CONDI 

SDtCOND 

14.00 

1 

0.00 

0.00 

9.00 

8 

0.00 

0.00 

2.00 

6 

0.00 

0.00 

2.00 

2 

-0.00 

0.00 

1.00 

4 

-0.00 

0.00 

2.00 

0 

-0.00 

0.00 

5.00 

1 

-0.00 

0.00 

1.00 

0 

-0.00 

0.00 

7.00 

1 

-0.00 

0.00 

5.00 

1 

-0.00 

0.00 

2.00 

1 

-0.00 

0.00 

2.00 

1 

-0.00 

0.00 

11.00 

2 

-0.00 

0.00 

22.00 

1 

-0.00 

0.00 

CHI-SQUARE=»»*««»***«» 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  12  DEGREES  OF  FREEDOM: 


21.0297 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  HEZ  SO 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K*0. 6542630+00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBDA=0.131550D+01 


TAU 

OBS 

EXPECTED (COND) 

SOCCONO 

1.00 

4 

1.32 

1.15 

4.00 

1 

3.60 

1.90 

2.00 

1 

1.11 

1.05 

6.00 

1 

1.83 

1.35 

3.00 

1 

0.44 

0.66 

21.00 

1 

0.70 

0.64 

CHI-SQUARE=0. 6596470+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM: 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  9 

THE  ESTIMATE  OF  B=  0.15750+00 
NUMBER  OF  ITERATIONS:  3 


TAU 

OBS 

EXPECTED (CONO» 

SD(COND) 

1.00 

4 

1.32 

1.15 

4.00 

1 

3.60 

1.90 

2.00 

1 

1.11 

1.05 

6.00 

1 

1.83 

1.35 

3.00 

1 

0.44 

0.66 

21.00 

1 

0.70 

0.64 

CHI-SQUARE* 

6. 

.5965 

0.950  QUANTILE  FOR  CHI-SCUARE  WITH  4  DEGREES  OF  FREEDOM* 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N:  16 

THE  ESTIMATE  OF  ALPHA=-0. 30980+00 
THE  ESTIMATE  OF  PHI*  0.196 70+00 
NUMBER  OF  ITERATIONS*  15 


9.4917 


9.4917 


TAU  OBS 

EXPECTEO(COND) 

SD(CONO) 

1.00 

4 

3.13 

1.77 

4.00 

1 

1.65 

1.29 

2.00 

1 

1.57 

1.25 

6.00 

1 

1.01 

1.00 

3.00 

1 

1.11 

1.05 

21.00 

1 

0.53 

0.73 

THE  CUMULATIVE 

NUMBER 

OF  ERRORS  OBSERVED=0. 9000000*01 

CHI-SQUARE* 
0.950  QUANTILE 

1.1372 
FOR  CHI 

-SQUARE  WITH 

3  DEGREES  OF  FREEDOM' 

IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 1711080*00 
NUMBER  OF  ITERATIONS*  6 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  10 


TAU 

OBS 

EXPECTED! COND ) 

SO (COND 

1.00 

4 

1.67 

1.29 

4.00 

1 

3.56 

1.89 

2.00 

1 

1.17 

1.08 

6.00 

1 

1.86 

1.36 

3.00 

1 

0.75 

0.87 

21.00 

1 

0.73 

0.86 

CHI-SQUARE*0. 5702990*01 

0.950  QUANTILE  FOR  CHI-SqUARE  WITH  4  DEGREES  OF  FREEDOM* 


BINOMIAL 


THE  ESTIMATE  OF  N=  7 

THE  ESTIMATE  OF  A*  0.51360*00 


TAU  OBS  EXPECTED (CONDI  SO(CONO) 


1.00 

4 

2.99 

1.73 

4.00 

1 

3.00 

1.73 

2.00 

1 

1.57 

1.25 

6.00 

1 

1.38 

1.17 

3.00 

1 

0.35 

0.59 

21.00 

1 

-0.56 

0.75 

CHI-SQUARE*  -1.1557 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM* 


7.8167 


9.4917 


9.4917 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  NEZ  ST 


GEOMETRIC  POISSON 

THE  ESTIMATE  OF  K*0. 6346740*00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBDAS.  1176110*01 


TAU 

OBS 

EXPECTED (CONO I 

SO (CONO 

1.00 

1 

1.18 

1.09 

3.00 

3 

2.49 

1.56 

1.00 

0 

0.57 

0.76 

5.00 

2 

1.72 

1.31 

12.00 

1 

1.04 

1.02 

CHI-SQUARE=0 . 7506410*00 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DE6REES  OF  FREEDOM: 


JELINSKI-HGRANOA 


NONHOMuGENEOUS  POISSON 


THE  ESTIMATE  OF  N-  7 

THE  ESTIMATE  OF  B:  0.18050*00 
NUMBER  OF  ITERATIONS:  3 


TAU 

OBS 

EXPECTED ( CONO 1 

SO (CONO 

1.00 

1 

1.18 

1.09 

3.00 

3 

2.49 

1.56 

1.00 

0 

0.57 

0.76 

5.00 

2 

1.72 

1.31 

12.00 

1 

1.04 

1.02 

CHI-SQUARE:  0.7506 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DECREES  OF  FREEDOM: 


GENERALIZED  POISSON 

IBM  POISSON 

THE  ESTIMATE  OF  PHI=0. 1353990*00 
NUMBER  OF  ITERATIONS:  6 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUSS*  8 

TAU  OBS  EXPECTED! COND )  S0( CONDI 


7.6167 


7.6167 


r 


i 


ts 


1.00 

1  1.12 

1.06 

3.00 

3  2.23 

1.49 

1.00 

0  0.72 

0.65 

5.00 

2  1.71 

1.31 

12.00 

1  1.06 

1.04 

CHI-SQUARE=0 

.1053160*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 

BINOMIAL 

3  DEGREES  C 

THE  ESTIMATE 

OF  N=  7 

THE  ESTIMATE 

OF  As  0.19750*00 

TAU  1 

DBS  EXPECTED ( COND) 

soccono: 

1.00 

1  1.27 

1.13 

3.00 

3  2.73 

1.65 

1.00 

0  0.56 

0.75 

5.00 

2  1.95 

1.40 

12.00 

1  1.00 

1.00 

CHI-SQUAREs 

0.6437 

7.6167 


0.950  QUANTILE  FOR  CHI-SQUARE  WITH  3  DEGREES  OF  FREEDOMS 


7.6167 


-N 


i 


APS  MEZ  IN 
APS  HTZ  IT 


IBM  POISSON  WITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


r- 


; 


£ 


r. 


K. 


THE  ESTIMATE  OF  K=0. 1013420*01 
NUMBER  OF  ITERATIONS1  5 

THE  ESTIMATE  OF  LAMBDA=0. 2671350*00 


CHI-5QUARE=0. 1080560*02 

0.950  QUANTILE  FOR  CHI-SQUARE  MITH 


JELINSKI-MORANDA 


6  DEGREES  OF  FREEDOMS 


12.5961 


A-54 


'  £jL 


.*  *.  •  '  •,*  •  '.v.V- 

Cm  •  l  i  »6Vi  '-Jj 


TAU 

OBS 

EXPECTED (COND) 

SO(CONO) 

P 

5.00 

1 

1.37 

1.17 

■) 

7.00 

1 

2.06 

1.44 

-V 

5.00 

2 

1.61 

1.27 

1.00 

2 

0.34 

0.56 

3.00 

2 

1.03 

1.02 

•»  * 

1.00 

0 

0.35 

0.59 

6.00 

2 

2.22 

1.49 

y 

5.00 

1 

1.99 

1.41 

*  *■ 


»  *  ^  mJT.  i 


HONHOMOGENEOUS  POISSON 

THE  ESTIMATE 

OF  N* 

-19 

THE  ESTIMATE 

OF  B*-0. 

13330-01 

NUMBER  OF  ITERATIONS* 

3 

TAU  1 

OBS 

EXPECTED < CONO) 

SO ( CONO ) 

5.00 

1 

1.37 

1.17 

7.00 

1 

2.08 

1.44 

5.00 

2 

1.61 

1.27 

1.00 

2 

0.34 

0.58 

3.00 

2 

1.03 

1.02 

1.00 

0 

0.35 

0.59 

6.00 

2 

2.22 

1.49 

5.00 

1 

1.99 

1.41 

CHI-SQUARE* 

10.8058 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6 

DEGREES  OF  FREEDOM’ 

GENERALIZED  POISSON 

THE  ESTIMATE 

OF  N= 

153 

THE  ESTIMATE 

OF  ALPHA 

=  0.14930*00 

THE  ESTIMATE 

OF  FH1* 

0. 7735D-02 

NUMBER  OF  ITERATIONS* 

13 

TAU  OBS 

EXPECTED (CONO) 

SO(COND) 

5.00 

1 

1.50 

1.23 

7.00 

1 

1.57 

1.25 

5.00 

2 

1.48 

1.22 

1.00 

2 

1.15 

1.07 

3.00 

2 

1.33 

1.15 

1.00 

0 

1.12 

1.06 

6.00 

2 

1.45 

1.21 

5.00 

1 

1.39 

1.18 

THE  CUMULATIVE  NUMBER 

OF  ERRORS  O6SERVE0*0. 1100000*02 

CHI-SQUARE*  2.9590 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM*  11.0733 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*0. 5553560-04 
NUMBER  OF  ITERATIONS*  11 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  3898 


55 


TAU 

OBS 

EXPECTEOC  COND ) 

SO (COND 

5.00 

1 

1.67 

1.29 

7.00 

1 

2.34 

1.53 

5.00 

2 

1.67 

1.29 

1.00 

2 

0.33 

0.58 

3.00 

2 

1.00 

1.00 

1.00 

0 

0.33 

0.58 

6.00 

2 

2.00 

1.41 

5.00 

1 

1.66 

1.29 

CHI-SQUARE*0 . 110256D+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM*  12.5961 


BINOMIAL 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  HTZ  SD 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  *=0.9006410+00 
NUMBER  OF  ITERATIONS*  4 

THE  ESTIMATE  OF  LAMBDA*0.220806D«01 


TAU  OBS  EXPECTED (COND)  SD(COND) 


7.00 

7 

11.54 

3.40 

1.00 

7 

1.06 

1.03 

3.00 

4 

2.59 

1.61 

4.00 

1 

2.40 

1.55 

6.00 

1 

2.16 

1.47 

1.00 

0 

0.25 

0.50 

CHI-SQUARE=0 . 3746390+02 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH  4  DEGREES  OF  FREEDOM*  9.4917 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N*  22 
THE  ESTIMATE  OF  B*  0.1046D+00 
NUMBER  OF  ITERATIONS*  3 


TAU  OBS  EXPECTED (COND)  SD(COND) 


CHI -SQUARES  37.4639 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


4  DE6REES  OF  FREEDOM2 


9.4917 


6ENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  16 
THE  ESTIMATE  OF  ALPHA=-0. 30810+00 
THE  ESTIMATE  OF  PHI=  0.11650+01 
NUf'iBER  OF  ITERATIONS2  9 


TAU 

0BS 

EXPECTEO ( COND  ) 

SO (COND 

7.00 

7 

10.41 

3.23 

1.00 

7 

11.97 

3.46 

3.00 

4 

3.55 

1.89 

4.00 

1 

0.21 

0.46 

6.00 

1 

-1.82 

1.35 

1.00 

0 

-4.33 

2.08 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVEDsO. 2000000+02 


CHI-SQUARE2  -2.5733 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


3  DEGREES  OF  FREEDOM2 


7.8167 


IBM  POISSON 


THE  ESTIMATE  OF  PHI2-. 4081540-04 
NUMBER  OF  ITERATIONS2  11 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS2  -11390 


Si 

TAU 

OBS 

EXPECTEO (COND) 

SO(COND) 

1 

7.00 

7 

6.36 

2.52 

L-\ 

1.00 

7 

0.91 

0.95 

3.00 

4 

2.73 

1.65 

4.00 

1 

3.64 

1.91 

6.00 

1 

5.46 

2.34 

l.CC 

0 

0.91 

0.95 

CHI-SQUARE=0. 4796580+02 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH 


4  DEGREES  OF  FREEDOM2 


9.4917 


BINOMIAL 


•  -  -  v  V  •.■.V.V.V.V.V.  -v- .  - .  -  .  •  -  - .  - « * .  -  .  -  V*  V  V  ■  '  .  ■  . 


•  -  _  • 


THE  ESTIMATE  OF  N=  16 
THE  ESTIMATE  OF  A=  0. 73250+00 


TAU  OBS  EXPECTED! COHO )  SO! COHO) 

7.00  7  16.39  4.05 

1.00  7  4.93  2.22 

3.00  4  2.22  1.49 

4.00  1  -1.43  1.19 

6.00  1  -2.48  1.57 

1.00  0  -1.82  1.35 

CHI-SQUARE=  -3.1396 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM:  9.4917 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  HTZ  ST 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 925117D+00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBOA=0. 1224710+01 


TAU 

OBS 

EXPECTED (CONO) 

SO (CONO 

1.00 

1 

1.22 

1.11 

1.00 

1 

1.13 

1.06 

2.00 

2 

2.02 

1.42 

2.00 

1 

1.73 

1.31 

3.00 

2 

2.14 

1.46 

6.00 

5 

3.03 

1.74 

7.00 

2 

2.14 

1.46 

10.00 

1 

1.60 

1.26 

CHI-SQUARE=0 . 1885810+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM:  12.5961 


JELINSKI-MORANDA 


THE  ESTIMATE  OF  N=  17 
THE  ESTIMATE  OF  ALPHA:  0.10000+01 
THE  ESTIMATE  OF  PHI=  0.70480-01 
NUMBER  OF  ITERATIONS:  7 


TAU 

OBS 

EXPECTED! CONO) 

SOI  CONO) 

1.00 

1 

1.17 

1.08 

1.00 

1 

1.10 

1.05 

2.00 

2 

1.92 

1.38 

CHI-SQUARE:  0.5711 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM: 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N:  16 

THE  ESTIMATE  OF  B=  0.77890-01 
NUMBER  OF  ITERATIONS:  2 


TAU 

OBS 

EXPECTED (CONDI 

SDICONO 

1.00 

1 

1.22 

1.11 

1.00 

1 

1.13 

1.06 

2.00 

2 

2.02 

1.42 

2.00 

1 

1.73 

1.31 

3.00 

2 

2.14 

1.46 

6.00 

5 

3.03 

1.74 

7.00 

2 

2.14 

1.46 

10.00 

1 

1.60 

1.26 

CHI-SQUARE: 

1 

.8850 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM: 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  16 
THE  ESTIMATE  OF  ALPHA:  0.12530*01 
THE  ESTIMATE  OF  PHI=  0.56670-01 
NUMBER  OF  ITERATIONS:  9 


TAU  OBS 

EXPECTED (COND) 

SO(COND) 

1.00 

1 

0.91 

0.95 

1.00 

1 

0.85 

0.92 

2.00 

2 

1.75 

1.32 

2.00 

1 

1.48 

1.22 

3.00 

2 

2.24 

1.50 

6.00 

5 

4.81 

2.19 

7.00 

2 

1.94 

1.39 

10.00 

1 

1.01 

1.00 

THE  CUMULATIVE 

NUMBER 

OF  ERRORS  OBSERVEDsO . 150000D*02 

CHI-SQUARE= 
0.950  QUANTILE 

0.2643 
FOR  CHI 

-SQUARE  HITH 

5  DEGREES  OF  FREEDOM: 

IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 6978680-01 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS= 


TAU 

OBS 

EXPECTED (CONO) 

SOtCOND 

1.00 

1 

1.21 

1.10 

1.00 

1 

1.14 

1.07 

2.00 

2 

1.94 

1.39 

2.00 

1 

1.67 

1.29 

3.00 

2 

2.22 

1.49 

6.00 

5 

3.66 

1.91 

7.00 

2 

1.74 

1.32 

10.00 

1 

1.23 

1.11 

CHI-SQUARE=0.920383D+00 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


BINOMIAL 


THE  ESTIMATE  OF  N=  17 
THE  ESTIMATE  OF  A=  0.80530-01 


6  DEGREES  OF  FREEDOM: 


12.5961 


TAU 

OBS 

EXPECTED! COND) 

SOtCOND 

1.00 

1 

1.28 

1.13 

1.00 

1 

1.20 

1.10 

2.00 

2 

2.17 

1.47 

2.00 

1 

1.87 

1.37 

3.00 

2 

2.48 

1.58 

6.00 

5 

3.66 

1.91 

7.00 

2 

1.97 

1.40 

10.00 

1 

1.42 

1.19 

CHI-SQUARE:  1.2172 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


6  DEGREES  OF  FREEDOM: 


12.5961 


APS  HTZ  IN 
APS  MIZ  IT 


IBM  POISSON  WITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0.102842D*01 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBDAS. 289863D+00 


vv.  -v-v s-w -r-.-  -v  .w 

.  -. •-  ••.  -  -  >.  .  >  ■  WV  -.V.' *.*V  v\  ■ 


m  m 


TAU 

OBS 

EXPECTED (COND) 

SO  (COND 

10.00 

2 

3.30 

1.82 

2.00 

0 

0.78 

0.88 

8.00 

5 

3.59 

1.69 

l.CS 

0 

0.51 

0.71 

3.00 

4 

1.61 

1.27 

2.00 

3 

1.15 

1.07 

3. CO 

2 

1.85 

1.36 

4.00 

0 

2.73 

1.65 

2.00 

1 

1.48 

1.22 

CHI-SQUARE=0 . 1175330*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DE6REES  OF  FREEDOM5  14.0702 


JE  LINSKI-MORAND A 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  H-  -9 
THE  ESTIMATE  OF  B=-0. 28030-01 
NUMBER  OF  ITERATIONS5  3 


TAU 

OBS 

EXPECTED (COM)) 

SD ( COND 

10.00 

2 

3.30 

1.82 

2.00 

0 

0.78 

0.88 

a. 00 

5 

3.59 

1.89 

1.00 

0 

0.51 

0.71 

3.00 

4 

1.61 

1.27 

2.00 

3 

1.15 

1.07 

3.00 

2 

1.85 

1.36 

4.00 

0 

2.73 

1.65 

2.00 

1 

1.48 

1.22 

CHI-SQUARE5  11.7533 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM5  14.0702 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 3100010-04 
NUMBER  OF  ITERATIONS5  10 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS5  8451 


TAU 


OBS 


EXPECTED (COND) 


SD(COND) 


0 


2.00 

0 

0.97 

0.99 

8.00 

5 

3.89 

1.97 

1.00 

0 

0.49 

0.70 

3.00 

4 

1.46 

1.21 

2.00 

3 

0.97 

0.99 

3.00 

2 

1.46 

1.21 

4.00 

0 

1.94 

1.39 

2.00 

1 

0.97 

0.99 

CHI-SQUARE*0. 1428270*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM* 


BINOMIAL 


THE  ESTIMATE  OF  N=  4 

THE  ESTIMATE  OF  A*-0.3767D-12 


TAU 

OBS 

EXPECTED (CONO) 

SO (CONO 

10.00 

2 

-0.00 

0.00 

2.00 

0 

-0.00 

0.00 

8.00 

5 

-0.00 

0.00 

1.00 

0 

0.00 

0.00 

3.00 

4 

0.00 

0.00 

2.00 

3 

0.00 

0.00 

3.00 

2 

0.00 

0.00 

4.00 

0 

0.00 

0.00 

2.00 

1 

0.00 

0.00 

CHI-SQUARE************ 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM* 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  MIZ  SO 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9371440*00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMBDA=0. 1746240*01 


TAU 

OBS 

EXPECTED (CONO) 

SDICOND 

5.00 

6 

7.70 

2.77 

4.00 

6 

4.59 

2.14 

1.00 

0 

0.97 

0.99 

4.00 

3 

3.32 

1.82 

2.00 

1 

1.36 

1.17 

2.00 

0 

1.20 

1.09 

5.00 

7 

2.39 

1.55 

1.00 

0 

0.39 

0.63 

1.00 

1 

0.37 

0.61 

%.  \  .v.-.  .1-  •  v  \- 


14.0702 


14.0702 


CHI-SQUARE*0. 1551870+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  10  DEGREES  OF  FREEDOM-  18.3111 


JE  LINSK I -HORANO A 


THE  ESTIMATE  OF  N=  28 
THE  ESTIMATE  OF  ALPHA*  0. 10000+01 
THE  ESTIMATE  OF  PHI=  0.56490-01 
NUMBER  OF  ITERATIONS*  12 


TAU 

OBS 

EXPECTED ( COND ) 

SO (COND 

5.00 

6 

7.94 

2.82 

4.00 

6 

4.77 

2.18 

1.00 

0 

1.02 

1.01 

4.00 

3 

3.41 

1.85 

2.00 

1 

1.59 

1.26 

2.00 

0 

1.48 

1.22 

5.00 

7 

3.14 

1.77 

1.00 

0 

0.51 

0.72 

1.00 

1 

0.35 

0.59 

2.00 

0 

0.46 

0.68 

5.00 

2 

0.88 

0.94 

23.00 

1 

1.44 

1.20 

CHI-SQUARE* 

12 

.1115 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  10  DEGREES  OF  FREEDOM*  18.3111 


NONHOHOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  28 
THE  ESTIMATE  OF  B=  0.64920-01 
NUMBER  OF  ITERATIONS*  2 


TAU  OBS  EXPECTED (CONDI  SO(CONO) 


CHI-SQUARE=  15.5187 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  10  DEGREES  OF  FREEDOMS  18.3111 


GENERALIZED  POISSON 

APS  MIZ  ST 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 8429760+00 
NUMBER  OF  ITERATIONS1  4 

THE  ESTIMATE  OF  LAMB0A=0. 2383410+01 


TAU 

OBS 

EXPECTED (COND ) 

1.00 

4 

2.38 

1.00 

1 

2.01 

1.00 

2 

1.69 

2.00 

2 

2.63 

3.00 

2 

2.59 

1.00 

1 

0.61 

1.00 

1 

0.51 

4.00 

0 

1.36 

12.00 

2 

1.21 

CHI-SQUARE=0 .453956D+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SO(CONO) 

1.54 

1.42 

1.30 

1.62 

1.61 

0.78 

0.72 

1.17 

1.10 


7  DEGREES  OF  FREEDOMS  14.0702 


JELINSKI-MORANDA 


NONHOMOGENE0US  POISSON 


THE  ESTIMATE  OF  N=  15 
THE  ESTIMATE  OF  B=  0. 17080+00 
NUMBER  OF  ITERATIONSs  3 


TAU 

OBS 

EXPECTED  (COND) 

SDtCOND 

1.00 

4 

2.38 

1.54 

1.00 

1 

2.01 

1.42 

1.00 

2 

1.69 

1.30 

2.00 

2 

2.63 

1.62 

3.00 

2 

2.59 

1.61 

1.00 

1 

0.61 

0.78 

1.00 

1 

0.51 

0.72 

4.00 

0 

1.36 

1.17 

12.00 

2 

1.21 

1.10 

CHI-SQUARE=  4.5396 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  OEGREES  OF  FREEDOM*  14.0702 


A'64 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  16 
THE  ESTIMATE  OF  ALPHA3  0.51920*00 
THE  ESTIMATE  OF  PHI=  0.16950*00 
NUMBER  OF  ITERATIONS1  6 


TAU  OBS 

EXFECTED(COND) 

SO(CONO) 

1.00 

4 

2.76 

1.66 

1.00 

1 

2.25 

1.50 

1.00 

2 

1.91 

1.38 

2.00 

2 

2.01 

1.42 

3.00 

2 

1.89 

1.37 

1.00 

1 

0.90 

0.95 

1.00 

1 

0.73 

0.85 

4.00 

0 

1.14 

1.07 

12.00 

2 

1.41 

1.19 

THE  CUMULATIVE 

NUMBER  OF  ERRORS  OBSERVE030. 1500000*02 

CHI-SQUARE3 

2. 

7711 

0.950  QUANTILE 

FOR 

CHI-SQUARE  WITH 

6  OEGREES  OF  FREEDOM' 

IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 1645430*00 
NUMBER  OF  ITERATIONS3  4 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS3  16 
APS  DAZ  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9472060*00 
NUMBER  OF  ITERATIONS3  5 

THE  ESTIMATE  OF  LAMBDA=0. 3290750*01 


TAU  OBS  EXPECTEO(CONO)  SO(CONO) 


1.00 

0 

0.61 

0.78 

1.00 

0 

0.58 

0.76 

13.00 

3 

5.27 

2.29 

3.00 

0 

0.77 

0.88 

4.00 

1 

0.85 

0.92 

1.00 

0 

0.19 

0.43 

CHI-SQUARE*0 . 31606 70*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  15  OEGREES  OF  FREEDOM* 


JELINSKI-MORANDA 


THE  ESTIMATE  OF  N=  62 
THE  ESTIMATE  OF  ALPHA*  0.10000*01 
THE  ESTIMATE  OF  PHI*  0.97030-01 
NUMBER  OF  ITERATIONS*  4 


TAU 

OBS 

EXPECTED! CONO) 

SO (COND 

2.00 

3 

5.85 

2.42 

1.00 

0 

2.88 

1.70 

6.00 

14 

16.42 

4.05 

1.00 

0 

2.50 

1.58 

3.00 

7 

6.66 

2.58 

5.00 

18 

10.15 

3.19 

1.00 

2 

1.33 

1.15 

1.00 

0 

0.90 

0.95 

2.00 

5 

1.71 

1.31 

4.00 

3 

2.67 

1.63 

5.00 

3 

2.16 

1.47 

1.00 

0 

0.38 

0.62 

1.00 

0 

0.34 

0.58 

13.00 

3 

3.78 

1.94 

3.00 

0 

0.59 

0.77 

4.00 

1 

0.60 

0.77 

1.00 

0 

0.10 

0.32 

CHI-SQUARE*  22.9963 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH  15  DEGREES  OF  FREEDOM* 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  62 
THE  ESTIMATE  OF  B*  0.54240-01 
NUMBER  OF  ITERATIONS*  2 


24.9997 


24.9997 


TAU 


OBS 


EXPECTED ( COND) 


SO(CONO) 


CHI-SQUAR£=  31.6067 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  15  DE6REES  OF  FREEDOM 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  57 
THE  ESTIMATE  OF  ALPHA:  0.15240*01 
THE  ESTIMATE  OF  PHI=  0.28990-01 
NUMBER  OF  ITERATIONS:  15 


TAU 

OBS 

EXPECTED <  COND ) 

SO (COND 

2.00 

3 

4.79 

2.19 

1.00 

0 

1.64 

1.28 

6.00 

14 

23.78 

4.88 

1.00 

0 

1.41 

1.19 

3.00 

7 

6.57 

2.56 

5.00 

18 

12.96 

3.60 

1.00 

2 

0.68 

0.83 

1.00 

0 

0.42 

0.65 

2.00 

5 

1.13 

1.06 

4.00 

3 

2.28 

1.51 

5.00 

3 

1.51 

1.23 

1.00 

0 

0.10 

0.32 

1.00 

0 

0.07 

0.27 

13.00 

3 

2.16 

1.47 

3.00 

0 

-0.08 

0.28 

4.00 

1 

-0.36 

0.60 

1.00 

0 

-0.07 

0.27 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED=0.590000D*02 


CHI-SQUARE=  22.9346 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


14  DEGREES  OF  FREEDOM 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 4898660-01 
NUMBER  OF  ITERATIONS:  5 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS= 


63 


TAU 

OBS 

EXPECTED! CONO) 

SO (CONO 

2.00 

3 

6.02 

2.45 

1.00 

0 

3.03 

1.74 

6.00 

14 

15.34 

3.92 

1.00 

0 

2.64 

1.63 

3.00 

7 

6.71 

2.59 

5.00 

18 

9.76 

3.12 

1.00 

2 

1.42 

1.19 

1.00 

0 

0.98 

0.99 

2.00 

5 

1.81 

1.35 

4.00 

3 

2.72 

1.65 

5.00 

3 

2.21 

1.49 

1.00 

0 

0.44 

0.66 

1.00 

0 

0.39 

0.62 

13.00 

3 

3.33 

1.82 

3.00 

0 

0.69 

0.83 

4.00 

1 

0.72 

0.85 

1.00 

0 

0.14 

0.38 

CHI-SOU ARE=0 . 2322930+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  15  DEGREES  OF  FREEDOM: 


BINOMIAL 


THE  ESTIMATE  OF  N=  64 
THE  ESTIMATE  OF  A=  0.52080-01 


TAU  OBS  EXPECTED!  CONDI  SO  (COMO 


2.00 

3 

6.38 

2.53 

1.00 

0 

3.12 

1.77 

6.00 

14 

16.50 

4.06 

1.00 

0 

2.41 

1.55 

3.00 

7 

6.87 

2.62 

5.00 

18 

9.28 

3.05 

1.00 

2 

1.14 

1.07 

1.00 

0 

1.04 

1.02 

2.00 

5 

2.03 

1.42 

4.00 

3 

2.91 

1.71 

5.00 

3 

2.86 

1.69 

1.00 

0 

0.48 

0.69 

1.00 

0 

0.48 

0.69 

13.00 

3 

4.67 

2.16 

3.00 

0 

0.94 

0.97 

4.00 

1 

1.22 

1.11 

1.00 

0 

0.28 

0.53 

CHI-SQUAREs 

24.7596 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  15  OEGREES  OF  FREEDOMS 


IBM  POISSON  WITH  VARIABLE  ALPHA 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9290140*00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBDA*0. 2166900+01 


TAU 

OBS 

EXPECTEO(CONO) 

SO(CONO) 

2.00 

3 

4.22 

2.05 

7.00 

9 

10.72 

3.27 

2.00 

3 

2.16 

1.46 

2.00 

4 

1.68 

1.37 

4.00 

4 

3.02 

1.74 

5.00 

2 

2.72 

1.65 

1.00 

0 

0.43 

0.66 

1.00 

2 

0.40 

0.63 

1.00 

0 

0.37 

0.61 

6.00 

2 

1.75 

1.32 

18.00 

1 

2  31 

1.52 

CHI-SQUARE*0 . 1177090+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


9  DEGREES  OF  FREEDOM* 


16.9252 


JELINSKI-MORANOA 


THE  ESTIMATE  OF  N=  31 
THE  ESTIMATE  OF  ALPHA*  0.10000+01 
THE  ESTIMATE  OF  PHI=  0. 52050-01 
NUMBER  OF  ITERATIONS*  20 


TAU 

OBS 

EXPECTED! CONO) 

SOtCOND) 

2.00 

3 

3.26 

1.61 

7.00 

9 

11.10 

3.33 

2.00 

3 

2.34 

1.53 

2.00 

4 

2.24 

1.50 

4.00 

4 

3.64 

1.91 

5.00 

2 

3.51 

1.67 

1.00 

0 

0.55 

0.74 

1.00 

2 

0.49 

0.70 

1.00 

0 

0.39 

0.62 

6.00 

2 

1.09 

1.04 

18.00 

1 

1.38 

1.16 

CHI-SQUARE*  9.0968 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


9  DEGREES  OF  FREEDOM*  16.9252 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N* 


A-69 


THE  ESTIMATE  OF  B=  0.73630-01 
NUMBER  OF  ITERATIONS:  2 


EXPECTED! COMO )  SO(CONO) 


CHI-SQUARE:  11.7709 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


9  DEGREES  OF  FREEDOM: 


16.9252 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  33 
THE  ESTIMATE  OF  ALPHA:  0. 65430*00 
THE  ESTIMATE  OF  PHI=  0.7765D-01 
NUMBER  OF  ITERATIONS:  6 


EXPECTED!  COND)  SDICOND) 


THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED*0. 3000000*02 


CHI-SQUARE:  5.3252 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


8  DEGREES  OF  FREEDOM:  15.5118 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0.733781D-01 
NUMBER  OF  ITERATIONS:  4 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS= 


kkkkkk  k  > 
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TAU 

oes 

EXPECTED! CONDI 

SD(C0ND 

2.00 

3 

4.09 

2.02 

7.00 

9 

11.55 

3.40 

2.00 

3 

2.82 

1.68 

2.00 

4 

2.68 

1.64 

9.00 

4 

3.92 

1.98 

5.00 

2 

3.46 

1.86 

1.00 

0 

0.58 

0.76 

1.00 

2 

0.51 

0.71 

1.00 

0 

0.36 

0.60 

6.00 

2 

0.34 

0.58 

18.00 

1 

-0.80 

0.89 

CHI-SQUARE*0 . 1150220*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


9  DEGREES  OF  FREEDOM*  16.9252 


BINOMIAL 


THE  ESTIMATE  OF  N=  32 
THE  ESTIMATE  OF  A=  0.6038D-01 


TAU 

OBS 

EXPECTED! CONO  I 

SO! COND 

2.00 

3 

3.59 

1.89 

7.00 

9 

9.83 

3.14 

2.00 

3 

2.22 

1.49 

2.00 

4 

1.88 

1.37 

4.00 

4 

2.69 

1.64 

5.00 

2 

2.22 

1.49 

1.00 

0 

0.38 

0.62 

1.00 

2 

0.38 

0.62 

1.00 

0 

0.27 

0.51 

6.00 

2 

1.38 

1.17 

18.00 

1 

1.67 

1.29 

CHI-SQUARE* 

11. 

5401 

0.950  QUANTILE  FOR 

1  CHI-SQUARE  WITH 

9  DEGREES  1 

IBM 

POISSON  WITH  VARIABLE  ALPHA 

APS  DAZ  ST 

GEOMETRIC  POISSON 

THE  ESTIMATE 

OF  K= 

0.9268520*00 

NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE 

OF  LAMBOAsO. 2118830*01 

TAU 

OBS 

EXPECTED! COND) 

SO! COND 

2.00 

5 

4.08 

2.02 

16.9252 


v  V  ••  -•  . .  - 
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0 

3 

3 


1.8; 

3.2! 

4.0' 


1.00 

0 

1.15 

1.07 

3.00 

6 

2.98 

1.73 

2.00 

2 

1.64 

1.28 

2.00 

1 

1.41 

1.19 

1.00 

2 

0.63 

0.79 

4.00 

3 

2.09 

1.44 

1.00 

0 

0.43 

0.66 

4.00 

1 

1.43 

1.19 

4.00 

0 

1.05 

1.03 

CHI-SQUARE*0. 1172990*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  11  DEGREES  OF  FREEDOM*  19.6806 


JE  LINSKI-MOR AND A 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  29 
THE  ESTIMATE  OF  B=  0.7596D-01 
NUMBER  OF  ITERATIONS*  2 


TAU 

OBS 

EXPECTED ( CONO ) 

SDICOND 

2.00 

5 

4.08 

2.02 

1.00 

0 

1.82 

1.35 

2.00 

3 

3.25 

1.80 

3.00 

3 

4.04 

2.01 

1.00 

0 

1.15 

1.07 

3.00 

6 

2.98 

1.73 

2.00 

2 

1.64 

1.28 

2.00 

1 

1.41 

1.19 

1.00 

2 

0.63 

0.79 

4.00 

3 

2.09 

1.44 

1.00 

0 

0.43 

0.66 

4.00 

1 

1.43 

1.19 

4.00 

0 

1.05 

1.03 

CHI-SQUARE*  11.7299 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  11  DEGREES  OF  FREEDOM*  19.6806 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*0 .8040470-01 
NUMBER  OF  ITERATIONS*  5 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS* 


27 


TAU 


OBS 


EXPECTED (COND) 


SDICOND) 


2.00 

5 

4.15 

2.04 

1.00 

0 

1.92 

1.39 

2.00 

3 

3.37 

1.84 

3.00 

3 

4.42 

2.10 

1.00 

0 

1.36 

1.16 

3.00 

6 

3.53 

1.88 

2.00 

2 

1.98 

1.41 

2.00 

1 

1.37 

1.17 

1.00 

2 

0.63 

0.79 

4.00 

3 

1.67 

1.29 

1.00 

0 

0.31 

0.56 

4.00 

1 

0.81 

0.90 

4.00 

0 

0.24 

0.49 

CHI-SQUAR£:0. 1040380*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  11  DEGREES  OF  FREEDOM:  19.0806 


BINOMIAL 


THE  ESTIMATE  OF  N=  29 
THE  ESTIMATE  OF  A=  0.76540-01 


TAU 

OBS 

EXPECTED 1 COND ) 

SD(  COND) 

2.00 

5 

4.18 

2.05 

1.00 

0 

1.80 

1.34 

2.00 

3 

3.47 

1.86 

3.00 

3 

4.40 

2.10 

1.00 

0 

1.36 

1.17 

3.00 

6 

3.79 

1.95 

2.00 

2 

1.77 

1.33 

2.00 

1 

1.49 

1.22 

1.00 

2 

0.70 

0.84 

4.00 

3 

1.97 

1.40 

1.00 

0 

0.33 

0.57 

4.00 

1 

1.18 

1.09 

4.00 

0 

0.91 

0.96 

CHI-SQUARE= 

9. 

.5557 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  11  DEGREES  OF  FREEDOM:  19.6806 


IBM  POISSON  HITH  VARIABLE  ALPHA 


APS  DAZ  IN 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9386410*00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMB0A=0. 8005170*00 


A-73 


v'vV. 


TAU 

OBS 

EXPECTEO!  CONDI 

SDtCONO 

2.00 

2 

1.55 

1.25 

2.00 

0 

1.37 

1.17 

2.00 

1 

1.20 

1.10 

2.00 

2 

1.06 

1.03 

4.00 

3 

1.76 

1.33 

1.00 

0 

0.37 

0.61 

2.00 

0 

0.68 

0.83 

CHI-SQUARE9!). 4292880+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


5  DEGREES  OF  FREEDOM9 


11.0733 


JE  LINSKI-MOR ANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N9  13 
THE  ESTIMATE  OF  B9  0. 6332D-01 
NUMBER  OF  ITERATIONS9  2 


TAU 

OBS 

EXPECTED (COND) 

SI)  (COND 

2.00 

2 

1.55 

1.25 

2.00 

0 

1.37 

1.17 

2.00 

1 

1.20 

1.10 

2.00 

2 

1.06 

1.03 

4.00 

3 

1.76 

1.33 

1.00 

0 

0.37 

0.61 

2.00 

0 

0.68 

0.83 

CHI-SQUARE9 

4.2929 

0.950  QUANTILE  FOR  CHI 

-SQUARE  WITH  5 

DEGREES  1 

GENERALIZED  POISSON 

THE  ESTIMATE 

OF  N9 

5 

THE  ESTIMATE 

OF  ALPHA9 

-0.22710*01 

THE  ESTIMATE 

OF  PHI9  0 

.47810+01 

NUMBER  OF  ITERATIONS9 

7 

TAU  i 

DBS 

EXPECTED! COND) 

SDtCONO 

2.00 

2 

5.34 

2.31 

2.00 

0 

4.35 

2.08 

2.00 

1 

2.37 

1.54 

2.00 

2 

1.38 

1.17 

4.00 

3 

0.08 

0.28 

1.00 

0 

-2.92 

1.71 

2.00 

0 

-2.59 

1.61 

11.0733 


A-74 


} 
i 
i 

! 

i 
5 
I 
1| 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED=0. 6000000+01 


CHI-SQUARE:  108.6003 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  4  DEGREES  OF  FREEDOM:  9.4917 


IBM  POISSON 

THE  ESTIMATE  OF  PHI=0. 8998430-01 
NUMBER  OF  ITERATIONS:  9 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS:  IQ 


TAU 

OBS 

EXPECTED! COND) 

SDICOND 

2.00 

2 

1.73 

1.31 

2.00 

0 

1.56 

1.25 

2.00 

1 

1.21 

1.10 

2.00 

2 

1.04 

1.02 

4.00 

3 

1.59 

1.26 

1.00 

0 

0.37 

0.60 

2.00 

0 

0.35 

0.59 

CHI-SQUAREsO. 4469970+01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  5  DEGREES  OF  FREEDOM:  11.0733 


BINOMIAL 


THE  ESTIMATE  OF  N=  16 
THE  ESTIMATE  OF  A:  0.52340-01 


TAU  OBS 


EXPECTED (CONDI  SO(COND) 


2.00 

2 

1.60 

2.00 

0 

1.40 

2.00 

1 

1.40 

2.00 

2 

1.30 

4.00 

3 

2.10 

1.00 

0 

0.41 

2.00 

0 

o.ro 

1.26 

1.16 

1.18 

1.14 

1.45 

0.64 

0.90 


CHI-SQUARE=  3.5982 

0.950  QUANTILE  FOR  CHI-SCi  /'PE  WITH  5  DEGREES  OF  FREEDOM:  11.0733 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  SAD  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0.938426D+00 
NUMBER  OF  ITERATIONS:  5 


THE  ESTIMATE  OF  LAMBDA=0.316891D+01 


TAU 

DBS 

EXPECTED <COND> 

SDCCONO 

1.00 

6 

3.17 

1.78 

1.00 

8 

2.97 

1.72 

1.00 

1 

2.79 

1.67 

3.00 

4 

7.38 

2.72 

7.00 

4 

12.62 

3.55 

5.00 

6 

6.13 

2.48 

2.00 

3 

1.96 

1.40 

2.00 

2 

1.72 

1.31 

2.00 

7 

1.52 

1.23 

3.00 

4 

1.94 

1.39 

2.00 

2 

1.10 

1.05 

1.00 

0 

0.50 

0.71 

1.00 

1 

0.47 

0.69 

20.00 

1 

5.16 

2.27 

5.00 

1 

0.55 

0.74 

CHI-SQUARE=0.477485D+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEDOM2 


22.3668 


JELINSKI-MORANDA 


THE  ESTIMATE  OF  N=  51 
THE  ESTIMATE  OF  ALPHA2  0.10000+01 
THE  ESTIMATE  OF  PHI2  0.5813D-01 
NUMBER  OF  ITERATIONS2  3 


TAU 

OBS 

EXPECTED! COND) 

SOtCONO 

1.00 

6 

2.99 

1.73 

1.00 

8 

2.88 

1.70 

1.00 

1 

2.24 

1.50 

3.00 

4 

6.36 

2.52 

7.00 

4 

12.81 

3.58 

5.00 

6 

8.28 

2.88 

2.00 

3 

2.73 

1.65 

2.00 

2 

2.27 

1.51 

2.00 

7 

2.15 

1.47 

3.00 

4 

2.35 

1.53 

2.00 

2 

0.99 

0.99 

1.00 

0 

0.44 

0.66 

1.00 

1 

0.20 

0.45 

20.00 

1 

2.89 

1.70 

5.00 

1 

0.43 

0.66 

CHI-SQUARE2  39.1583 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEDOM2 


22.3668 


NONHOMOGENEOUS  POISSON 


1  >  V 


•  .N  ' 

.  >  .  -  V  -  V 

o-  - c  v"  •  ■  • 


V  V’ 


,\V. 


THE  ESTIMATE  OF  N=  51 
THE  ESTIMATE  OF  B=  0.63550-01 
NUMBER  OF  ITERATIONS:  2 


TAU 

OBS 

EXPECTED  ICONO) 

1.00 

6 

3.17 

1.00 

8 

2.97 

1.00 

1 

2.79 

3.00 

4 

7.38 

7.00 

4 

12.62 

5.00 

6 

6.13 

2.00 

3 

1.96 

2.00 

2 

1.72 

2.00 

7 

1.52 

3.00 

4 

1.94 

2.00 

2 

1.10 

1.00 

0 

0.50 

1.00 

1 

0.47 

20.00 

1 

5.16 

5.00 

1 

0.55 

CHI-SQUARE=  47.7485 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SO(CONO) 

1.78 

1.72 
1.67 

2.72 
3.55 
2.48 
1.40 
1.31 
1.23 
1.39 
1.05 
0.71 
0.69 
2.27 
0.74 


DEGREES  OF  FREEDOM:  22.3668 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  M«  57 
THE  ESTIMATE  OF  ALPHA:  0.18740+00 
THE  ESTIMATE  OF  PHI:  0.1044D+00 
NUMBER  OF  ITERATIONS:  7 


TAU 

OBS 

EXPECTED (COMO) 

SD(COMD) 

1.00 

6 

5.94 

2.44 

1.00 

8 

5.73 

2.39 

1.00 

1 

4.59 

2.14 

3.00 

4 

5.38 

2.32 

7.00 

4 

5.55 

2.36 

5.00 

6 

4.79 

2.19 

2.00 

3 

3.44 

1.85 

2.00 

2 

2.96 

1.72 

2.00 

7 

2.84 

1.69 

3.00 

4 

2.43 

1.56 

2.00 

2 

1.65 

1.29 

1.00 

0 

1.35 

1.16 

1.00 

1 

0.93 

0.96 

20.00 

1 

1.45 

1.20 

5.00 

1 

0.98 

0.99 

THE  CUMULATIVE  NUMBER  OF  ERRORS  0BSERVE0=0. 5000000*02 


CHI-SQUARE:  13.8245 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  12  DEGREES  OF  FREEDOM:  21.0297 


A-77 


»  t  1fl\  f nt  m  *** 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 6657750-01 
NUMBER  OF  ITERATIONS:  6 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS= 


TAU 

OBS 

EXPECTED (COND) 

SD(COND 

1.00 

6 

3.46 

1.86 

1.00 

8 

3.33 

1.82 

1.00 

1 

2.60 

1.61 

3.00 

4 

6.91 

2.63 

7.00 

4 

12.24 

3.50 

5.00 

6 

6.45 

2.91 

2.00 

3 

3.09 

1.76 

2.00 

2 

2.57 

1.60 

2.00 

7 

2.44 

1.56 

3.00 

4 

2.61 

1.62 

2.00 

2 

1.16 

1.08 

1.00 

0 

0.53 

0.73 

1.00 

1 

0.27 

0.51 

20.00 

1 

2.23 

1.49 

5.00 

1 

0.58 

0.76 

CHI-SQUARE=0 . 304158D+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEDOM* 


BINOMIAL 


22.3666 


THE  ESTIMATE  OF  N=  51 
THE  ESTIMATE  OF  As  0.4415D-01 


TAU 

OBS 

EXPECTED ( COND ) 

SDCCOND 

1.00 

6 

2.20 

1.48 

1.00 

8 

1.94 

1.39 

1.00 

1 

1.60 

1.26 

3.00 

4 

4.47 

2.11 

7.00 

4 

8.51 

2.92 

5.00 

6 

5.55 

2.36 

2.00 

3 

1.86 

1.36 

2.00 

2 

1.61 

1.27 

2.00 

7 

1.44 

1.20 

3.00 

4 

1.24 

1.11 

2.00 

2 

0.51 

0.71 

1.00 

0 

0.17 

0.42 

1.00 

1 

0.17 

0.42 

20.00 

1 

1.76 

1.33 

5.00 

1 

0.40 

0.63 

CHI-SQUARE=  66.3159 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEOOMs 


22.3666 


.N. 


IBM  POISSON  UITH  VARIABLE  ALPHA 


SAD  SO 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9075340*00 
NUMBER  OF  ITERATIONS2  5 

THE  ESTIMATE  OF  LAMBDA=0. 4836 920+01 


TAU 

OBS 

EXPECTED (COND) 

SOtCONO! 

1.00 

3 

4.84 

2.20 

2.00 

6 

8.37 

2.89 

1.00 

0 

3.62 

1.90 

1.00 

6 

3.28 

1.81 

1.00 

2 

2.98 

1.73 

1.00 

2 

2.70 

1.64 

4.00 

16 

8.53 

2.92 

2.00 

6 

3.17 

1.78 

2.00 

4 

2.61 

1.62 

3.00 

1 

3.08 

1.76 

3.00 

1 

2.30 

1.52 

1.00 

0 

0.63 

0.79 

1.00 

1 

0.57 

0.76 

1.00 

1 

0.52 

0.72 

6.00 

1 

2.25 

1.50 

8.00 

1 

1.54 

1.24 

CHI-SQUARE-0.219550D+02 

0.950  QUANTILE  FOR  CHI-SQUARE  MXTH  14  DEGREES  OF  FREEDOM2 


JELINSKI-MORANDA 


23.6908 


THE  ESTIMATE  OF  N2  53 
THE  ESTIMATE  OF  ALPHA2  0.10000+01 
THE  ESTIMATE  OF  PHI2  0.70880-01 
NUMBER  OF  ITERATIONS2  10 


TAU 

OBS 

EXPECTED (COND) 

SDCCOND 

1.00 

3 

3.74 

1.93 

2.00 

6 

7.34 

2.71 

1.00 

0 

3.60 

1.90 

1.00 

6, 

3.39 

1.84 

1.00 

2 

3.32 

1.82 

1.00 

2 

3.17 

1.78 

4.00 

16 

10.71 

3.27 

2.00 

6 

3.51 

1.87 

2.00 

4 

3.09 

1.76 

3.00 

1 

3.14 

1.77 

3.00 

1 

2.50 

1.58 

•\  -  .  •*.  *•. 
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^  /  “  ’,*■  .*  * 
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1.00  0  0.55 
1.00  1  0.41 

1.00  1  0.34 

6.00  1  1.18 

8.00  1  1.01 


CHI-SQUARE3  16.6943 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


0.74 

0.64 

0.58 

1.09 

1.00 


14  DEGREES  OF  FREEDOMS  23.6908 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  52 
THE  ESTIMATE  OF  B=  0.9702D-01 
NUMBER  OF  ITERATIONS:  3 


OBS  EXPECTED! COND)  SD(COND) 


1.00  3  4.84 
2.00  6  8.37 
1.00  0  3.62 
1.00  6  3.28 
1.00  2  2.98 
1.00  2  2.70 
4.00  16  8.53 
2.00  6  3.17 
2.00  4  2.61 
3.00  1  3.08 
3.00  1  2.30 
1.00  0  0.63 
1.00  1  0.57 
1.00  1  0.52 
6.00  1  2.25 
8.00  1  1.54 


CHI-S9UARE=  21.9550 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 

GENERALIZED  POISSON 


2.20 

2.89 

1.90 
1.81 
1.73 
1.64 
2.92 
1.78 
1.62 
1.76 
1.52 
0.79 
0.76 
0.72 
1.50 
1.24 


14  DEGREES  OF  FREEDOM:  23.6908 


THE  ESTIMATE  OF  N=  52 
THE  ESTIMATE  OF  ALPHA3  0.13000+01 
THE  ESTIMATE  OF  PHI=  0.5926D-01 
NUMBER  OF  ITERATIONS3  7 


TAU  OBS  EXPECTED! COND)  SDICOND ) 

1-00  3  3.09  1.76 
2*00  6  7.46  2.73 
1.00  0  2.97  1.72 
1-00  6  2.79  1^67 
1-00  2  2.73  1.65 
1.00  2  2.61  1.62 
4-00  16  13.33  3.65 


THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED=0.510000D+02 


CHI-SQUARE=  17.1907 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEDOM* 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 9225290-01 
NUMBER  OF  ITERATIONS*  3 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  49 


TAU  OBS  EXPECTED (CONO)  SD(CONO) 


1.00 

3 

4.48 

2.12 

2.00 

6 

8.37 

2.89 

1.00 

0 

4.29 

2.07 

1.00 

6 

4.02 

2.00 

1.00 

2 

3.92 

1.98 

1.00 

2 

3.74 

1.93 

4.00 

16 

10.77 

3.28 

2.00 

6 

3.61 

1.90 

2.00 

4 

3.09 

1.76 

3.00 

1 

2.66 

1.63 

3.00 

1 

1.90 

1.33 

1.00 

0 

0.33 

0.57 

1.00 

1 

0.14 

0.38 

1.00 

1 

0.05 

0.22 

6.00 

1 

-0.65 

0.80 

a. oo 

1 

-1.33 

1.15 

CHI-SQUARE=0 . 2965040*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  14  DEGREES  OF  FREEDOM 


BINOMIAL 


THE  ESTIMATE  OF  N=  53 
THE  ESTIMATE  OF  A*  0.9776D-01 


TAU 


OBS 


EXPECTED! CONO) 


SO(COND) 


0 

4.12 

2.03 

6 

4.12 

2.03 

2 

3.56 

1.89 

2 

3.37 

1.64 

16 

11.07 

3.33 

6 

3.23 

1.80 

4 

2.17 

1.47 

1 

2.08 

1.44 

1 

1.63 

1.35 

CHI-SQUARE®  17.1500 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


14  DEGREES  OF  FREEDOM: 


23.6906 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  SAD  ST 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9280560+00 
NUMBER  OF  ITERATIONS:  5 


THE  ESTIMATE  OF  LAMBOA:0.169485D+Ol 


CHI-SQUARE=0 .1367760+02 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH 


JELINSKI-MORANDA 


7  DEGREES  OF  FREEDOM: 


14.0702 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  24 
THE  ESTIMATE  OF  B*  0.7466D-01 
NUMBER  OF  ITERATIONS®  2 


TAU 


DBS 


EXPECTED (CONO l 


50  (COMO) 


1.00 

0 

1.69 

1.00 

3 

1.57 

1.00 

0 

1.46 

3.00 

7 

3.78 

4.00 

2 

3.89 

5.00 

3 

3.48 

3.00 

4 

1.54 

2.00 

0 

0.85 

2.00 

0 

0.73 

CHI-SQUARE* 
0.950  QUANTILE 

13.6776 
FOR  CHI' 

-SQUARE  WITH 

1.30 

1.25 

1.21 

1.94 

1.97 

1.87 

1.24 

0.92 

0.86 


7  DEGREES  OF  FREEDOM:  14.0702 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  ALPHA*  0.13140*01 
THE  ESTIMATE  OF  PHI*  0.6396D-01 
NUMBER  OF  ITERATIONS*  8 


TAU 

OBS 

EXPECTED! COND 1 

SDICOND 

1.00 

0 

1.33 

1.15 

1.00 

3 

1.26 

1.12 

1.00 

0 

1.14 

1.07 

3.00 

7 

4.55 

2.13 

4.00 

2 

4.26 

2.06 

5.00 

3 

4.12 

2.03 

3.00 

4 

1.29 

1.14 

2.00 

0 

0.60 

0.77 

2.00 

0 

0.44 

0.66 

THE  CUMULATIVE  NUMBER  OF  ERRORS  0BSERVE0*0. 1900000*02 
CHI-SQUARE*  14.3756 

0.950  QUANTILE  FOR  CHI-SQUARE  NITH  6  DEGREES  OF  FREEDOM*  12.5961 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0.789144D-01 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  23 


l:-. 


TAU 

OBS 

EXPECTED! CONO) 

SO! CONO 

1.00 

0 

1.78 

1.34 

1.00 

3 

1.70 

1.31 

1.00 

0 

1.55 

1.24 

3.00 

7 

4.06 

2.02 

4.00 

2 

3.53 

1.88 

A-83 


A 


m 


5.00 

3 

3.23 

1.80 

3.00 

4 

1.44 

1.20 

2.00 

0 

0.85 

0.92 

2.00 

0 

0.70 

0.83 

CHI-SQUARE*0. 1321030+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  0E6REES  OF  FREEDOM*  14.0702 


BINOMIAL 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  A*  0.10540+00 


TAU 

OBS 

EXPECT EO ( COND ) 

soccond: 

1.00 

0 

2.08 

1.44 

1.00 

3 

2.08 

1.44 

1.00 

0 

1.78 

1.33 

3.00 

7 

4.82 

2.19 

4.00 

2 

3.71 

1.93 

5.00 

3 

3.59 

1.90 

3.00 

4 

1.57 

1.25 

2.00 

0 

0.34 

0.58 

2.00 

0 

0.34 

0.58 

CHI-SQUARE* 

10 

.5990 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM*  14.0702 


IBM  POISSON  WITH  VARIABLE  ALPHA 


APS  SAD  IN 
APS  ZBZ  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9382600+00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMBDA=0. 1314400+01 


TAU 

OBS 

EXPECTED (CONDI 

SOI COND 

1.00 

0 

1.31 

1.15 

5.00 

2 

5.45 

2.33 

10.00 

13 

6.85 

2.62 

1.00 

2 

0.47 

0.69 

3.00 

1 

1.25 

1.12 

1.00 

0 

0.37 

0.61 

3.00 

0 

0.97 

0.99 

6.00 

1 

1.47 

1.21 

14.00 

1 

1.86 

1.36 

CHI-SQUARE=0. 1587010+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  0EGREE3  OF  FREEOOM* 


14.0702 


JELINSKI-MORANOA 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  ALPHA5  0.10000*01 
THE  ESTIMATE  OF  PHI5  0.50000-01 
NUMBER  OF  ITERATIONS5  5 


TAU 

08S 

EXPECTED (CONO) 

SO (CONO 

1.00 

0 

1.07 

1.04 

5.00 

2 

5.11 

2.26 

10.00 

13 

9.72 

3.12 

1.00 

2 

0.62 

0.79 

3.00 

1 

0.96 

0.98 

1.00 

0 

0.27 

0.52 

3.00 

0 

0.51 

0.72 

6.00 

1 

0.73 

0.85 

14.00 

1 

0.99 

1.00 

CHI-SQUARE5  0.0144 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


7  OESREES  OF  FREEDOM 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N5  21 
THE  ESTIMATE  OF  B=  0.63730-01 
NUMBER  OF  ITERATIONS5  2 


TAU 

OBS 

EXPECTED (CONO) 

SD(COND 

1.00 

0 

1.31 

1.15 

5.00 

2 

5.45 

2.33 

10.00 

13 

6.85 

2.62 

1.00 

2 

0.47 

0.69 

3.00 

1 

1.25 

1.12 

1.00 

0 

0.37 

0.61 

3.00 

0 

0.97 

0.99 

6.00 

1 

1.47 

1.21 

14.00 

1 

1.86 

1.36 

CHI-SQUARE5  15.0701 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DE6REES  OF  FREEDOM 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  CF  ALPHA5  0.12040*01 
THE  ESTIMATE  OF  PHI5  0.29950-01 
NUMBER  OF  ITERATIONS5  14 


TAU 


063 


EXPECTED (CONDI 


SO(COND) 


1.00 

0 

0.64 

0.80 

5.00 

2 

4.79 

2.19 

10.00 

13 

11.09 

3.33 

1.00 

2 

0.37 

0.61 

3.00 

1 

0.77 

0.88 

1.00 

0 

0.16 

0.40 

3.00 

0 

0.40 

0.63 

6.00 

1 

0.68 

0.62 

14.00 

1 

1.12 

1.06 

THE  CUMULATIVE  NUMBER  OP  ERRORS  OBSERVED=0 . 2000000*02 


CHI-SQUARE*  10.6501 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DE6REES  OF  FREEDOM*  12.5961 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 5515590-01 
NUMBER  OF  ITERATIONS*  6 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  22 


TAU 

OBS 

EXPECTED (CONDI 

1.00 

0 

1.20 

5.00 

2 

5.12 

10.00 

13 

8.55 

1.00 

2 

0.70 

3.00 

1 

1.06 

1.00 

0 

0.32 

3.00 

0 

0.59 

6.00 

1 

0.79 

14.00 

1 

0.96 

CHI -SQUARE =0 .8773660*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


BINOMIAL 


THE  ESTIMATE  OF  N=  22 
THE  ESTIMATE  OF  A*  0.66020-01 


TAU 

OBS 

EXPECTEO(CONO) 

1.00 

0 

1.39 

5.00 

2 

6.10 

10.00 

13 

9.51 

1.00 

2 

0.43 

3.00 

1 

0.84 

1.00 

0 

0.24 

SD(CONO) 

1.10 

2.26 

2.92 

0.84 

1.03 

0.56 

0.77 

0.89 

0.98 


7  DEGREES  OF  FREEDOM*  14.0702 


SD( CONDI 

1.18 

2.47 

3.08 

0.65 

0.92 

0.49 


S-. 

■.rr:V.T\  V-  '  ‘  ~ :  5~  • .-  ■.'  •.' 

*. 

p 

J 

i 

•>: 

V. 

3.00  0 

0.66 

0.81 

- « 

6.00  1 

1.20 

1.10 

•>: 

14.00  1 

1.62 

1.27 

a 

CHI-SQUARE*  12.4096 

V 

0.950  QUANTILE  FOR  CHI 

-SQUARE  WITH 

7  DEGREES  OF  FREEDOM* 

14.0702 

;•> 

IBM  POISSON  MITH  VARIABLE  ALPHA 

■ 

APS  ZBZ  SO 

?»*■ 

> 

GEOMETRIC  POISSON 

THE  ESTIMATE  OF  K=0. 9409240*00 

NUMBER  OF  ITERATIONS* 

4 

V* 

THE  ESTIMATE  OF  LAMBDA 

*0.5114720*01 

■ 

TAU  OBS 

EXPECTED  (CONDI 

SO(CONO) 

•Iv 

3.00  1 

14.46 

3.80 

6.00  26 

22.07 

4.70 

1.00  4 

2.96 

1.72 

1.00  6 

2.78 

1.67 

2.00  10 

5.08 

2.25 

H 

4.00  13 

8.48 

2.91 

2.00  4 

3.53 

1.88 

v* 

1.00  4 

1.61 

1.27 

■"v 

2.00  4 

2.94 

1.71 

1.00  0 

1.34 

1.16 

\v 

1.00  2 

1.26 

1.12 

1.00  1 

1.19 

1.09 

CiL 

1.00  1 

1.12 

1.06 

m 

5.00  1 

4.67 

2.16 

w 

6.00  2 

4.01 

2.00 

% 

3.00  0 

1.52 

1.23 

CHI-SQUARE*0. 3571300*02 

£ 

0.950  QUANTILE  FOR  CHI 

-SQUARE  MITH 

14  0E6REES  OF  FREEDOM* 

23.6908 

i 

JELINSKI 

-MOR  AND  A 

THE  ESTIMATE  OF  N* 

85 

«v 

THE  ESTIMATE  OF  ALPHA* 

0.10000*01 

THE  ESTIMATE  OF  PHI*  0 

.51970-01 

y 

c2*t 

NUMBER  OF  ITERATIONS* 

10 

fl 

sV 

TAU  OBS 

EXPECTED (COND) 

SD(CONO) 

3.00  1 

13.31 

3.65 

V/ 

6.00  26 

26.00 

5.10 

1.00  4 

4.23 

2.06 

*  '  » 

V 

1.00  6 

3.61 

1.90 

V 

i 

S' 

*.  > 

2.00  10 

7.00 

2.65 

>: 

s 

»> 

I.J*. 

k> 

& 

A-87 

SK 

!? 

V-V- V-V-.% V-V-  V 

•  v 

■!-v ’S*-.' ‘ 

-  -  •  •  .  -  A.'., 

.Sv'AkS/S\>vVv'v  V 

*-  .*'  ■*'  r**  .  • 

*.*  s*  \  *  •  Oa*  % 

•  .  •  .  -V-  .  •  A  .V.N 

•.  .*  v*  -  O -.1 

CHI -SQUARE5  26.4206 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  14  DEGREES  OF  FREEDOM* 


22.6906 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  67 
THE  ESTIMATE  OF  B=  0.60890-01 
NUMBER  OF  ITERATIONS*  2 


TAU 

OBS 

EXPECTED (COND) 

SOI  COND 

3.00 

1 

14.46 

3.80 

6.00 

26 

22.07 

4.70 

1.00 

4 

2.96 

1.72 

1.00 

6 

2.78 

1.67 

2.00 

10 

5.06 

2.25 

4.00 

13 

8.48 

2.91 

2.00 

4 

3.53 

1.68 

1.00 

4 

1.61 

1.27 

2.00 

4 

2.94 

1.71 

1.00 

0 

1.34 

1.16 

1.00 

2 

1.26 

1.12 

1.00 

1 

1.19 

1.09 

1.00 

1 

1.12 

1.06 

5.00 

1 

4.67 

2.16 

6.00 

2 

4.01 

2.00 

3.00 

0 

1.52 

1.23 

CHI-SQUARE*  35.7130 

0.950  QUANTILE  FOR  CHI-SQUARE  NITH  14  DEGREES  OF  FREEDOM* 


GENERALIZED  POISSON 


23.6906 


THE  ESTIMATE  OF  N*  85 
THE  ESTIMATE  OF  ALPHA*  0.71280*00 
THE  ESTIMATE  OF  PHI*  0.72200-01 
NUMBER  OF  ITERATIONS*  5 


TAU  0B3 


EXPECTED t COND) 


SD(COND) 


v-.  •%  -  v  w*  --  ----- 

aVaVV  V  ’.'.V 


/  s 


( 


THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED=0. 7900000*02 


CHI-SQUARE=  22.8874 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DE6REES  OF  FREEOOM= 


IBM  POISSON 


22.3668 


THE  ESTIMATE  OF  PHI=0. 5677420-01 
NUMBER  OF  ITERATIONS=  4 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS: 


TAU 

OBS 

EXPECTED (CONDI 

SDtCOND 

3.00 

1 

13.70 

3.70 

6.00 

26 

24.60 

4.96 

1.00 

4 

4.61 

2.15 

1.00 

6 

3.93 

1.98 

2.00 

10 

7.41 

2.72 

4.00 

13 

10.25 

3.20 

2.00 

4 

3.11 

1.76 

1.00 

4 

1.31 

1.15 

2.00 

4 

2.44 

1.56 

1.00 

0 

0.80 

0.90 

1.00 

2 

0.75 

0.86 

1.00 

1 

0.69 

0.83 

1.00 

1 

0.53 

0.76 

5.00 

1 

2.07 

1.44 

6.00 

2 

2.12 

1.45 

3.00 

0 

0.99 

0.99 

CHI-SQUARE-0 . 2630290+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  14  DE6REES  OF  FREEDOM: 


23.6908 


BINOMIAL 


THE  ESTIMATE  OF  N=  93 
THE  ESTIMATE  OF  A:  0.47620-01 


TAU 


OBS 


EXPECTED (COND) 


SD(COND) 


3.00 

1 

12.32 

3.51 

6.00 

26 

22.74 

4.77 

1.00 

4 

3.05 

1.75 

1.00 

6 

2.86 

1.69 

2.00 

10 

5.04 

2.25 

4.00 

13 

7.89 

2.81 

2.00 

4 

2.95 

1.72 

1.00 

4 

1.33 

1.15 

2.00 

4 

2.23 

1.49 

1.00 

0 

0.95 

0.98 

1.00 

2 

0.95 

0.98 

1.00 

1 

0.86 

0.93 

1.00 

1 

0.81 

0.90 

5.00 

1 

3.50 

1.87 

6.00 

2 

3.85 

1.96 

3.00 

0 

1.80 

1.34 

CHI-SQUARE* 

36.6033 

0.9S0  QUANTILE  FOR  CHI-SQUARE  WITH  14  DE6REES  OF  FREEDOM*  23.6908 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  ZBZ  ST 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0.927748D*00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMBDA=0 . 3412880*01 


TAU 

OBS 

EXPECTEO(CONO) 

SD(CQND 

1.00 

3 

3.41 

1.85 

1.00 

1 

3.17 

1.78 

1.00 

2 

2.94 

1.71 

1.00 

2 

2.73 

1.65 

2.00 

5 

4.87 

2.21 

3.00 

11 

6.07 

2.46 

1.00 

2 

1.74 

1.32 

1.00 

5 

1.61 

1.27 

4.00 

0 

5.37 

2.32 

3.00 

4 

3.09 

1.76 

4.00 

6 

3.17 

1.78 

5.00 

0 

2.84 

1.68 

CHI-SQUARE*0 . 2418080*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  10  DEGREES  OF  FREEDOM*  18.3111 


JELINSKI-MORANDA 


•9 


K 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N*  47 
THE  ESTIMATE  OF  B=  0. 75000-01 
NUMBER  OF  ITERATIONS*  2 


TAU 

OBS 

EXPECTED (CONO) 

SD(CONO 

1.00 

3 

3.41 

1.85 

1.00 

1 

3.17 

1.78 

1.00 

2 

2.94 

1.71 

1.00 

2 

2.73 

1.65 

2.00 

5 

4.87 

2.21 

3.00 

11 

6.07 

2.46 

1.00 

2 

1.74 

1.32 

1.00 

5 

1.61 

1.27 

4.00 

0 

5.37 

2.32 

3.00 

4 

3.09 

1.76 

4.00 

6 

3.17 

1.78 

5.00 

0 

2.84 

1.68 

CHI-SQUARE* 

24. 

1808 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  10  DEGREES  OF  FREEDOM*  IB. 3111 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0.B03159O-01 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  44 


TAU 

OBS 

EXPECTED (CONO) 

SD( CONO 

1.00 

3 

3.57 

1.89 

1.00 

1 

3.32 

1.82 

1.00 

2 

3.24 

1.80 

1.00 

2 

3.16 

1.78 

2.00 

5 

5.77 

2.40 

3.00 

11 

7.20 

2.68 

1.00 

2 

1.72 

1.31 

1.00 

5 

1.48 

1.22 

4.00 

0 

4.95 

2.23 

3.00 

4 

2.75 

1.66 

4.00 

6 

2.67 

1.64 

5.00 

0 

1.16 

1.08 

CHI-SQUARE*0. 2396900*02 

0.950  QUANTILE  FOR  CHI-SQUARE  MITH  10  DEGREES  OF  FREEOOM*  18.3111 


BINOMIAL 


THE  ESTIMATE  OF  N=  45 
THE  ESTIMATE  OF  A=  0. 87330-01 


TAU 

OBS 

EXPECTED (COND) 

SO  (COND 

1.00 

3 

3.74 

1.93 

1.00 

1 

3.49 

1.87 

1.00 

2 

3.40 

1.84 

1.00 

2 

3.24 

1.80 

2.00 

5 

5.88 

2.42 

3.00 

11 

7.30 

2.70 

1.00 

2 

1.73 

1.32 

1.00 

5 

1.56 

1.25 

4.00 

0 

4.04 

2.01 

3.00 

4 

3.16 

1.78 

4.00 

6 

2.86 

1.69 

5.00 

0 

1.31 

1.14 

CHI-SQUARE: 

21 

.5964 

0.950  QUANTILE  FOP  CHI-SQUARE  WITH  10  DEGREES  OF  FREEDOM:  18.3111 


IBM  POISSON  WITH  VARIABLE  ALPHA 

APS  ZBZ  IN 
DRS  DAD  IT 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9701360*00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAM8DA=0. 6420080*00 


TAU 

OBS 

EXPECTED! COND ) 

50  (COND) 

4.00 

1 

2.46 

1.57 

7.00 

1 

3.64 

1.91 

4.00 

1 

1.76 

1.33 

7.00 

11 

2.61 

1.62 

2.00 

0 

0.65 

0.31 

3.00 

1 

0.90 

0.95 

10.00 

1 

2.48 

1.57 

8.00 

0 

1.51 

1.23 

9.00 

1 

1.31 

1.15 

11.00 

2 

1.19 

1.09 

6.00 

0 

0.50 

0.71 

CHI-SQUARE=0 . 3428030*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOM:  16.9252 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  21 
THE  ESTIMATE  OF  B=  0.30320-01 
NUMBER  OF  ITERATIONS®  2 


TAU 

OBS 

EXPECTED (CONDI 

SO (CONDI 

4.00 

1 

2.46 

1.57 

7.00 

1 

3.64 

1.91 

4.00 

1 

1.76 

1.33 

7.00 

11 

2.61 

1.62 

2.00 

0 

0.65 

0.81 

3.00 

1 

0.90 

0.95 

10.00 

1 

2.48 

1.57 

8.00 

0 

1.51 

1.23 

9.00 

1 

1.31 

1.15 

11.00 

2 

1.19 

1.09 

6.00 

0 

0.50 

0.71 

CHI-SQUARE®  34.2803 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOM®  18.9252 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 306 7540-01 
NUMBER  OF  ITERATIONS®  4 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS®  21 


TAU 

OBS 

EXPECTED (CONDI 

4.00 

1 

2.46 

7.00 

1 

3.71 

4.00 

1 

2.10 

7.00 

11 

3.32 

2.00 

0 

0.96 

3.00 

1 

1.33 

10.00 

1 

1.59 

8.00 

0 

1.09 

9.00 

1 

0.97 

11.00 

2 

0.86 

6.00 

0 

0.33 

CHI-SQUARE=0 . 2538710+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SD(  CONDI 

1.57 

1.93 

1.45 

1.82 

0.98 

1.16 

1.26 

1.05 

0.98 

0.93 

0.58 


9  DEGREES  OF  FREEDOM®  16.9252 


BINOMIAL 


I 


A-93 


THE  ESTIMATE  OF  N=  20 
THE  ESTIMATE  OF  A=  0. 391X0-01 


TAU 

OBS 

EXPECTED ( COND ) 

SD ( COND 

4.00 

1 

2.91 

1.71 

7.00 

1 

4.57 

2.14 

4.00 

1 

2.62 

1.62 

7.00 

11 

4.09 

2.02 

2.00 

0 

0.46 

0.68 

3.00 

1 

0.67 

0.62 

10.00 

1 

1.64 

1.26 

8.00 

0 

1.10 

1.05 

9.00 

1 

1.21 

1.10 

11.00 

2 

1.08 

1.04 

6.00 

0 

0.23 

0.48 

CHI-SQUARE:  19.7285 

0.950  QUANTILE  FOR  CHI-SQUARE  MITH  9  DEGREES  OF  FREEDOM:  16.9252 


IBM  POISSON  WITH  VARIABLE  ALPHA 

DRS  DAD  SD 
DRS  DAD  ST 
DRS  DAD  IN 
OSS  LDR  IT 
OSS  LDR  SO 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 8714290*00 
NUMBER  OF  ITERATIONS:  5 

THE  ESTIMATE  OF  LAMBOA=0. 3503960*01 


TAU 

OBS 

EXPECTED (COND) 

SD(COND) 

2.00 

10 

6.56 

2.56 

1.00 

1 

2.66 

1.63 

1.00 

1 

2.32 

1.52 

6.00 

10 

8.83 

2.97 

2.00 

1 

1.66 

1.29 

7.00 

1 

3.23 

1.80 

3.00 

0 

0.67 

0.82 

4.00 

1 

0.56 

0.75 

5.00 

2 

0.38 

0.62 

2.00 

0 

0.09 

0.30 

1.00 

0 

0.04 

0.19 

CHI-SQUARE=0. 1364470*02 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH  9  DEGREES  OF  FREEDOM:  16.9252 

JELINSKI-MORANDA 


A-94 


N0NH0M0GENE0U3  POISSON 


THE  ESTIMATE  OF  N=  27 
THE  ESTIMATE  OF  B*  0.13760*00 
NUMBER  OF  ITERATIONS*  3 


TAU 

OBS 

EXPECTED (COND) 

SDtCOND 

2.00 

10 

6.56 

2.56 

1.00 

1 

2.66 

1.63 

1.00 

1 

2.32 

1.52 

6.00 

10 

8. S3 

2.97 

2.00 

1 

1.66 

1.29 

7.00 

1 

3.23 

1.80 

3.00 

0 

0.67 

0.62 

4.00 

1 

0.56 

0.75 

5.00 

2 

0.38 

0.62 

2.00 

0 

0.09 

0.30 

1.00 

0 

0.04 

0.19 

CHI-SQUARE* 

13. 

.6447 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  9  DEGREES  OF  FREEDOM* 


GENERALIZED  POISSON 


10.9252 


IBM  POISSON 


THE  ESTIMATE  OF  PHI*0. 1333900*00 
NUMBER  OF  ITERATIONS*  4 


THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS* 


TAU 

OBS 

EXPECTED! COND ) 

SD(COND) 

2.00 

10 

6.05 

2.46 

1.00 

1 

3.11 

1.76 

1.00 

1 

2.98 

1.72 

6.00 

10 

7.09 

2.66 

2.00 

1 

1.82 

1.35 

7.00 

1 

2.72 

1.65 

3.00 

0 

1.15 

1.07 

4.00 

1 

1.00 

1.00 

5.00 

2 

0.66 

0.82 

2.00 

0 

0.07 

0.27 

1.00 

0 

-0.36 

0.60 

CHI-SOU ARE=0 . 1151060*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


BINOMIAL 


9  DEGREES  OF  FREEDOM* 


16.9252 


V  *-•  ■ 


•  .V^v'vjvY*  .v. 

v .  V.  v*Vi 


THE  ESTIMATE  OF  N=  26 
THE  ESTIMATE  OF  A=  0.20640*00 


TAU 

OBS 

EXPECTED (COND) 

SO(COND) 

2.00 

10 

8.6B 

2.95 

1.00 

1 

2.92 

1.71 

1.00 

1 

2.73 

1.65 

6.00 

10 

9.70 

3.11 

2.00 

1 

1.24 

1.11 

7.00 

1 

2.03 

1.43 

3.00 

0 

0.77 

0.88 

4.00 

1 

0.93 

0.97 

5.00 

2 

0.43 

0.65 

2.00 

0 

-0.45 

0.67 

1.00 

0 

-0.25 

0.50 

CHI-SQUARE* 

9. 

0205 

0.950  QUANTILE  FOR 

CHI-SQUARE  WITH 

9  DE6REES  OF 

OSS  LOR  ST 

IBM  POISSON  WITH  VARIABLE  ALPHA 

GEOMETRIC  POISSON 

THE  ESTIMATE 

OF  K=0. 

1004180*01 

NUMBER  OF  ITERATIONS 

*  5 

THE  ESTIMATE 

OF  LAMBOA=0. 3135820*00 

TAU 

OBS 

EXPECTED (COND) 

SD(COND) 

5.00 

2 

1.58 

1.26 

4.00 

1 

1.29 

1.14 

5.00 

1 

1.64 

1.28 

3.00 

1 

1.00 

1.00 

2.00 

1 

0.67 

0.82 

1.00 

0 

0.34 

0.58 

8.00 

4 

2.77 

1.66 

2.00 

0 

0.71 

0.84 

CHI-SQUARE=0. 2178570*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  0E6REES  OF  FREEDOM* 


JELINSKI-MORANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  -74 
THE  ESTIMATE  OF  B=-0. 4X680-02 
NUMBER  OF  ITERATIONS*  2 


TAU 


06S 


EXPECTED ( CONO ) 


SO (CONO) 


5.00 

2 

1.58 

1.26 

4.00 

1 

1.29 

1.14 

5.00 

1 

1.64 

1.28 

3.00 

1 

1.00 

1.00 

2.00 

1 

0.67 

0.82 

1.00 

0 

0.34 

0.58 

8.00 

4 

2.77 

1.66 

2.00 

0 

0.71 

0.84 

CHI-SQUARE* 

2.1786 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM* 


12.5961 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0.721501D-04 
NUMBER  OF  ITERATIONS*  10 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  2762 


TAU 

DBS 

EXPECTED (COND) 

5.00 

2 

1.67 

4.00 

1 

1.33 

5.00 

1 

1.67 

3.00 

1 

1.00 

2.00 

1 

0.67 

1.00 

0 

0.33 

8.00 

4 

2.66 

2.00 

0 

0.67 

CHI-SQUARE*0. 2252920*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SOCCOND ) 

1.29 

1.16 

1.29 

1.00 

0.B2 

0.5S 

1.63 

0.82 


6  OEGREES  OF  FREEDOM*  12.5961 


BINOMIAL 


THE  ESTIMATE  OF  N=  -15 
THE  ESTIMATE  OF  A=-0. 17660-01 


TAU  08S  EXPECTED! CONO)  SO (CONO) 


i  LmlmAl 


2.00 


0 


0.95 


0.97 


fr.'. 


CHI-SQUARE=  2.1291 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  6  DEGREES  OF  FREEDOM:  12.5961 


IBM  POISSON  WITH  VARIABLE  ALPHA 


SES  VAS  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 9692020+00 
NUMBER  OF  ITERATIONS:  4 

THE  ESTIMATE  OF  LAMBOA=0. 6114530+00 


TAU 

OBS 

EXPECTED! CONO) 

SDtCOND 

7.00 

3 

3.90 

1.98 

1.00 

0 

0.49 

0.70 

7.00 

3 

3.04 

1.74 

2.00 

2 

0.75 

0.87 

1.00 

0 

0.36 

0.60 

1.00 

1 

0.35 

0.59 

1.00 

0 

0.34 

0.58 

4.00 

4 

1.25 

1.12 

4.00 

0 

1.10 

1.05 

6.00 

0 

1.42 

1.19 

CHI-SQUARE=0 . 1325790+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  8  DEGREES  OF  FREEDOM:  15.5118 


JE  LINSKI-MOR AND A 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  20 
THE  ESTIMATE  OF  B:  0.31280-01 
NUMBER  OF  ITERATIONS:  2 


TAU  OBS  EXPECTED! COND )  SO(COND) 


CHI-SQUARE=  13.2579 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  8  DEGREES  OF  FREEDOM' 


15.5118 


K 


GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0.348362D-04 
NUMBER  OF  ITERATIONS:  7 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS:  5231 


TAU 

OBS 

EXPECTED (COND) 

SOICONO 

7.00 

3 

2.68 

1.64 

1.00 

0 

0.38 

0.62 

7.00 

3 

2.68 

1.64 

2.00 

2 

0.77 

0.87 

1.00 

0 

0.38 

0.62 

1.00 

1 

0.38 

0.62 

1.00 

0 

0.38 

0.62 

4.00 

4 

1.53 

1.24 

4.00 

0 

1.53 

1.24 

6.00 

0 

2.29 

1.51 

CHI-SQUARE=0 . 1202990*02 

0.950  QUANTILE  FOR  CHI-SQUARE  HXTH  8  DEGREES  OF  FREEDOM:  15.5118 


BINOMIAL 


THE  ESTIMATE  OF  N:  19 

THE  ESTIMATE  OF  A=  0.31320-01 


TAU 

OBS 

EXPECTEO( COND ) 

SO (COND 

7.00 

3 

3.68 

1.92 

1.00 

0 

0.48 

0.70 

7.00 

3 

3.09 

1.76 

2.00 

2 

0.77 

0.8B 

1.00 

0 

0.33 

0.57 

1.00 

1 

0.33 

0.57 

1.00 

0 

0.30 

0.55 

4.00 

4 

1.14 

1.07 

4.00 

0 

0.67 

0.82 

6.00 

0 

0.98 

0.99 

CHI-SQUARE= 

13. 

3687 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  8  DEGREES  OF  FREEDOM:  15.5118 

IBM  POISSON  WITH  VARIABLE  ALPHA 


.1 

J 


SES  VAS  SO 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K*0. 1010800+01 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMBDA=0. 4495890+00 


TAU 

OBS 

EXPECTED (COND) 

2.00 

0 

0.90 

6.00 

4 

2.83 

1.00 

0 

0.49 

1.00 

1 

0.50 

6.00 

0 

3.09 

3.00 

5 

1.62 

2.00 

1 

1.11 

4.00 

1 

2.29 

4.00 

6 

2.39 

4.00 

1 

2.50 

2.00 

0 

1.29 

CHI-SQUARE*0. 2090730+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


SD(CONO) 

0.95 

1.68 

0.70 

0.70 

1.76 

1.27 

1.05 

1.51 

1.55 

1.58 

1.14 


9  DEGREES  OF  FREEDOM*  16.9252 


JE  LINSKI-MOR ANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N*  -41 
THE  ESTIMATE  OF  B=-0. 10740-01 
NUMBER  OF  ITERATIONS*  3 


TAU 

OBS 

EXPECTED ( COND ) 

2.00 

0 

0.90 

6.00 

4 

2.83 

1.00 

0 

0.49 

1.00 

1 

0.50 

6.00 

0 

3.09 

3.00 

5 

1.62 

2.00 

1 

1.11 

4.00 

1 

2.29 

4.00 

6 

2.39 

4.00 

1 

2.50 

2.00 

0 

1.29 

CHI-SQUARE* 

20. 

9073 

0.950  QUANTILE  FOR 

CHI-SQUARE  WITH 

SD(CONO) 

0.95 

1.68 

0.70 

0.70 

1.76 

1.27 

1.05 

1.51 

1.55 

1.58 

1.14 


9  DEGREES  OF  FREEDOM*  16.9TF2 


GENERALIZED  POISSON 


A-100 


IBM  POISSON 


THE  ESTIMATE  OF  PHI =0.1284110-03 
NUMBER  OF  ITERATIONS*  12 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS*  3268 


TAU 

OSS 

EXPECTED (COND) 

2.00 

0 

1.09 

6.00 

4 

3.26 

1.00 

0 

0.54 

1.00 

1 

0.54 

6.00 

0 

3.26 

3.00 

5 

1.63 

2.00 

1 

1.09 

4.00 

1 

2.17 

4.00 

6 

2.17 

4.00 

1 

2.16 

2.00 

0 

1.08 

CHI-SQUARE*0 . 2152920*02 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH 


SO(COND) 

1.04 

1.81 

0.74 

0.74 

1.81 

1.28 

1.04 

1.47 

1.47 

1.47 

1.04 


9  DEGREES  OF  FREEDOM*  16.9252 


BINOMIAL 


IBM  POISSON  HITH  VARIABLE  ALPHA 


SES  VAS  ST 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 6998580*00 
NUMBER  OF  ITERATIONS*  4 

THE  ESTIMATE  OF  LAMBDA=0. 1208740*01 


TAU 

OBS 

EXPECTED (CONDI 

SO (COND 

3.00 

2 

2.65 

1.63 

1.00 

1 

0.41 

0.64 

2.00 

1 

0.49 

0.70 

8.00 

0 

0.45 

0.67 

CHI-SQUARE*0 . 1953420*01 

0.950  QUANTILE  FOR  CHI-SQUARE  HITH  2  DEGREES  OF  FREEOOM*  5.9948 


JELINSKI-MCRANDA 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  4 

THE  ESTIMATE  OF  B=  0.3569D+00 
NUMBER  OF  ITERATIONS5  4 


TAU  OBS 

EXPECTED! CONO ) 

SO(CONO) 

3.00 

2 

2.65 

1.63 

1.00 

1 

0.41 

0.64 

2.00 

1 

0.49 

0.70 

8.00 

0 

0.45 

0.67 

CHI-SQUARE5 

1. 

9534 

0.950  QUANTILE 

FOR 

CHI-SQUARE  WITH 

2  DEGREES  OF 

GENERALIZED  POISSON 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 4242900*00 
NUMBER  OF  ITERATIONS5  17 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS5  4 


TAU 

OBS 

EXPECTED (CONO) 

sot  CONO 

3.00 

2 

2.85 

1.69 

1.00 

1 

1.07 

1.03 

2.00 

1 

0.35 

0.59 

8.00 

0 

-0.47 

0.69 

SES  VAS  IN 
SUS  CON  IT 


GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K=0. 1032400*01 
NUMBER  OF  ITERATIONS5  6 

THE  ESTIMATE  OF  LAMB0A=0 . 2161540-01 


5.9948 


TAU 


OBS 


EXPECTED(COND)  SD(COND) 


*•  .*• . 


CHI-SQUARE=0.907788D+02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEDOMS 


JELINSKI-MORANDA 


22.3668 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N=  0 

THE  ESTIMATE  OF  B=-0 . 31890-01 
NUMBER  OF  ITERATIONS3  4 


TAU 

OBS 

EXPECTED (CONDI 

SD(COND 

1.00 

1 

0.02 

0.15 

3.00 

1 

0.07 

0.26 

4.00 

0 

0.10 

0.32 

63.00 

3 

5.56 

2.36 

6.00 

3 

1.87 

1.37 

6.00 

0 

1.75 

1.32 

12.00 

1 

4.68 

2.16 

7.00 

5 

3.68 

1.92 

3.00 

1 

1 .85 

1.36 

1.00 

1 

0.66 

0.81 

2.00 

6 

1.38 

1.17 

3.00 

6 

2.24 

1.50 

1.00 

1 

0.79 

0.69 

1.00 

0 

0.82 

0.91 

4.00 

0 

3.55 

1.88 

CHI-SQUARE=  90.7788 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  13  DEGREES  OF  FREEDOM3 


GENERALIZED  POISSON 


22.3668 


IBM  POISSON 


THE  ESTIMATE  OF  PHI-0 . 367442D-04 
NUMBER  OF  ITERATIONS3  9 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BUGS3  3269 


TAU  OBS  EXPECTED (CONO)  SD(CONO) 
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F/G  9/2 


NL 


TAU 

06S 

EXPECTED! COND ) 

SO! COND) 

12.00 

3 

3. 98 

1.99 

5.00 

4 

2.07 

1.44 

1.00 

0 

0.45 

0.67 

5.00 

3 

2.43 

1.56 

2.00 

0 

1.07 

1.03 

CHI-SQUARE*0. 3680530*01 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


JELINSKI-MORANDA 


3  0E6REES  OF  FREEDOM* 


7.8167 


NONHOMOGENE OUS  POISSON 


THE  ESTIMATE  OF  N=  -9 
THE  ESTIMATE  OF  B*-0.2680D-01 
NUMBER  OF  ITERATIONS*  3 


TAU 

08S 

EXPECTED! COND) 

SOI  COND) 

12.00 

3 

3.98 

1.99 

5.00 

4 

2.07 

1.44 

1.00 

0 

0.45 

0.67 

5.00 

3 

2.43 

1.56 

2.00 

0 

1.07 

1.03 

CHI-SQUARE*  3.6605 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  04 
THE  ESTIMATE  OF  ALPHA*  0.91620*00 
THE  ESTIMATE  OF  PHI*  0.57670-02 
NUMBER  OF  ITERATIONS*  15 


3  DEGREES  OF  FREEDOM* 


7.8167 


TAU 

OBS 

EXPECTED! COND  I 

SOICONO 

12.00 

3 

4.71 

2.17 

5.00 

4 

2.09 

1.44 

1.00 

0 

0.47 

0.69 

5.00 

3 

1.93 

1.39 

2.00 

0 

0.80 

0.90 

THE  CUMULATIVE  NUMBER  OF  ERRORS  OBSERVED*0. 1000000*02 


CHI-SQUARE*  4.2390 

0.950  QUANTILE  FOR  CHZ-SQUAPE  WITH 


2  DEGREES  OF  FREEDOM* 


5. 9948 


IBM  POISSON 


THE  ESTIMATE  OF  PHI=0. 1131690-01 
NUMBER  OF  ITERATIONS-  4 

THE  ESTIMATE  OF  THE  NUM8ER  OF  INITIAL  BUGS*  39 


TAU 

OBS 

EXPECTED (CONO) 

SO  (CONO 

12.00 

3 

5.04 

2.25 

5.00 

4 

2.13 

1.46 

1.00 

0 

0.42 

0.65 

5.00 

3 

1.80 

1.34 

2.00 

0 

0.66 

0.81 

CHI-SQUARE*0.436152D+01 

0.950  QUANTILE  FOR  CHI-SQUARE  NITH  3  DE6REES  OF  FREEDOM*  7.8167 


BINOMIAL 


THE  ESTIMATE  OF  N=  2 

THE  ESTIMATE  OF  A*  0.26490-15 


TAU 

OBS 

EXPECTED (CONO) 

SO (CONO 

12.00 

3 

0.00 

0.00 

5.00 

4 

-0.00 

0.00 

1. 00 

0 

-0.00 

0.00 

5.00 

3 

-0.00 

0.00 

2.00 

0 

-0.00 

0.P0 

CHI-SQUARE************ 

0.950  QUANTILE  FOR  CHI-SQUARE  MITH  3  DECREES  OF  FREEDOM*  7.8167 


IBM  POISSON  NITH  VARIABLE  ALPHA 

SUS  CON  ST 

GEOMETRIC  POISSON 


THE  ESTIMATE  OF  K*0. 9496980+00 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  LAMB0A*0. 6040410+00 


TAU 

OBS 

EXPECTED (CONO) 

SO(CONO) 

4.00 

0 

2.24 

1.50 

5.00 

5 

2.22 

1.49 

5.00 

0 

1.72 

1.31 
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i 


5.00  1  0.04 

*.00  0  0.49 

1.00  2  0.25 

1.00  1  0.22 

1.00  0  0.20 

16.00  2  2.17 

10.00  0  0.60 


0.91 

0.70 

0.40 

0.46 

0.45 

1.47 

0.03 


CHI-SQUAREsO. 25594 90*02 

0.950  QUANTILE  FOR  CHI-SqUARE  WITH  0  0E6REES  OF  FREEDOMS 


JELINSKI-MORANDA 


15.5110 


THE  ESTIMATE  OF  N*  12 

THE  ESTIMATE  OF  ALPHAS  0.10000*01 
THE  ESTIMATE  OF  PHI®  0.44710-01 
NUMBER  OF  ITERATIONS®  5 

I 


TAU 

OBS 

EXPECTED (CONO) 

SO( CONO 

4.00 

0 

2.17 

1.47 

5.00 

5 

2.49 

1.56 

5.00 

0 

2.04 

1.43 

3.00 

1 

1.09 

1.04 

2.00 

0 

0.64 

0.60 

1.00 

2 

0.23 

0.46 

1.40 

1 

0.16 

0.43 

1.00 

0 

0.14 

0.37 

16.00 

2 

1.52 

1.23 

10.00 

0 

0.50 

0.71 

CHI-SQUAREs 

25.4756 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  0  0E6REES  OF  FREEDOMS  15.5110 


NONHOMOGENEOUS  POISSON 


THE  ESTIMATE  OF  N®  12 
THE  ESTIMATE  OF  B*  0.51610-01 
MJMBER  OF  ITERATIONS®  2 


TAU  OBS  EXPECTED (CONO)  SO  (CONO) 


4.00  0 

5.00  5 

5.00  0 

5.00  1 

*.00  0 

1.00  2 

1.00  1 

1.00  0 

16.00  2 

10.00  0 


2.24 

*.2* 

1.72 

0.64 

0.49 

0.25 

0.22 

0.20 

2.17 

0.66 


1.50 
1.49 

1.51 
0.91 
0.70 
0.48 
0.46 
0.45 
1.47 
0.63 
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CHI-SQUAI»E=  25.5949 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  0  DECREES  OF  FREEDOM:  15.5110 


GENERALIZED  POISSON 


THE  ESTIMATE  OF  N=  16 
THE  ESTIMATE  OF  ALPHA5  0.31520*00 
THE  ESTIMATE  OF  PHI5  0.71630-01 
NUMBER  OF  ITERATIONS5  7 


TAU 

OBS 

EXPECTED! CONDI 

SO!  CONDI 

4.00 

0 

1.00 

1.34 

5.00 

5 

1.61 

1.35 

5.00 

0 

1.57 

1.25 

3.00 

1 

1.24 

1.11 

2.00 

0 

1.00 

1.00 

1.00 

2 

0.66 

0.01 

1.00 

1 

0.59 

0.77 

1.00 

0 

0.51 

0.72 

16.00 

2 

1.06 

1.03 

10.00 

0 

0.77 

0.00 

THE  CUMULATIVE  NUMBER  OF  ERRORS  0BSERVE050. 1100000*02 

CHI-SQUARE5  15.1733 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  7  DEGREES  OF  FREEDOM5  14.0702 


IBM  POISSON 


THE  ESTIMATE  OF  PHI50. 4430150-01 
NUMBER  OF  ITERATIONS*  5 

THE  ESTIMATE  OF  THE  NUMBER  OF  INITIAL  BU6S5  13 


TAU 

OBS 

EXPECTED 1 CONDI 

SOICOND 

4.00 

0 

2.13 

1.46 

5.00 

5 

2.40 

1.55 

5.00 

0 

1.99 

1.41 

3.00 

1 

1.12 

1.06 

2.00 

0 

0.60 

0.02 

1.00 

2 

0.26 

0.51 

1.00 

1 

0.21 

0.46 

1.00 

0 

0.17 

0.41 

16.00 

2 

1.45 

1.20 

10.00 

0 

0.66 

0.61 

CHI-SQUARE*0. 2333770*02 

0.950  QUANTILE  FOR  CHI-SQUARE  WITH  0  DEGREES  OF  FREEDOM5  15.5110 


BINOMIAL 
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GLOSSARY 


Deliverable  Source  Lines  of  Code 

Implementation  Phase 

Design  Verification  Phase 

Computer  Program  Configuration  Item 

Computer  Program  Component 

Compilation  Unit 

Application  Set 

Diagnostic  Set 

Data  Reduction  Set 

Operating  System  Set 

System  Exercise  Set 

Support  Set 

System  Control  Set 

On-Site  Verification 

Software  Engineering  Division 

In  Accordance  With 

Operational  Test  and  Evaluation 

Program  Trouble  Report 

Program  Change  Report 

List  of  Affected  Modules 

Program  Support  Library 

Software  Change  Review  Board 

Hierarchical  Input,  Process,  Output 

Formal  Qualification  Verification 


S\  (X,Y)  -  9 g(X, Y)  ,  g2  (X, Y)  -  3g(X, Y) 

3X  3Y 


and  (Xq,  Y0)  is  an  initial  guess  at  (X,  Y) .  As  before,  if  (Xp, 
Yp)  is  close  enough  to  (X,Y),  the  sequence  (Xn,Yn)  converges  to 
(X,Y)  as  n  becomes  larger. 

This  technique  can  be  generalized  to  any  number  of  equa¬ 
tions  and  unknowns.  Also,  derivative  approximations  can  be  used 
in  place  of  exact  derivatives;  e.g.: 

Hihl 1  * 

8X 

where  h  is  small. 
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*) 


MISSION 

of 

Rome  Air  Development  Center 

RAVC  plant  and  executes  KM,  touch,  development,  tut  and 
selected  acquisition  pAogAams  in  suppoAt  o £  Command,  ContKol 
Communications  and  Intelligence  (C3I J  activities.  Technical 
and  englneeKing  suppont  uiithin  ajieas  ofi  technical  competence 
is  pKovided  to  ESV  PKogKam  0 6 faces  (POs)  and  otheA  ESV 
elements.  The  principal  technical  mission  asieas  axe 
communications,  electKomagnetlc  guidance  and  contAol,  sua- 
veillance  o&  gAound  and  aeAospace  objects.  Intelligence  data 
collection  and  handling,  InfaKmatlon  system  technology, 
lonospheAlc  pAopagatlon,  solid  state  sciences,  mlcAovoave 
physics  and  electAonic  Aellabillty,  maintainability  and 
compatibility. 


